͑i(4SSKJr SSKJr /r"SS5rg)) text_format) data_feed_pb2c6\rSrSrSrSrSrSrSrSr Sr g ) DataFeedDescat :api_attr: Static Graph Datafeed descriptor, describing input training data format. DataFeedDesc shall be initialized from a valid protobuf message from disk. See :code:`paddle/base/framework/data_feed.proto` for message definition. A typical message might look like: Examples: .. code-block:: python >>> import paddle.base as base >>> with open("data.proto", "w") as f: ... f.write('name: "MultiSlotDataFeed"\n') ... f.write('batch_size: 2\n') ... f.write('multi_slot_desc {\n') ... f.write(' slots {\n') ... f.write(' name: "words"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write(' slots {\n') ... f.write(' name: "label"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write('}') >>> data_feed = base.DataFeedDesc('data.proto') However, users usually shouldn't care about the message format; instead, they are encouraged to use :code:`Data Generator` as a tool to generate a valid data description, in the process of converting their raw log files to training files acceptable to Executor. DataFeedDesc can also be changed during runtime. Once you got familiar with what each field mean, you can modify it to better suit your need. E.g.: .. code-block:: python >>> import paddle.base as base >>> data_feed = base.DataFeedDesc('data.proto') >>> data_feed.set_batch_size(128) >>> data_feed.set_dense_slots(['words']) # The slot named 'words' will be dense >>> data_feed.set_use_slots(['words']) # The slot named 'words' will be used >>> # Finally, the content can be dumped out for debugging purpose: >>> print(data_feed.desc()) Args: proto_file(string): Disk file containing a data feed description. c[R"5UlSURl[ US5n[ R "UR5UR5 SSS5 URRS:XaO[URRR5VVs0sHup4URU_M snnUl gg!,(df  Nx=fs snnf)NcatrMultiSlotDataFeed) rr proto_desc pipe_commandopenrParsereadname enumeratemulti_slot_descslots_DataFeedDesc__name_to_index)self proto_filefislots Z/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/base/data_feed_desc.py__init__DataFeedDesc.__init__Qs'446',$ *c "a   affh 8# ??  #6 6 ))H)H)N)NO$OGA 1 O$D  7# "$s0C6C+ C(c$XRlg)a| Set :attr:`batch_size` in ``paddle.base.DataFeedDesc`` . :attr:`batch_size` can be changed during training. Examples: .. code-block:: python >>> import paddle.base as base >>> with open("data.proto", "w") as f: ... f.write('name: "MultiSlotDataFeed"\n') ... f.write('batch_size: 2\n') ... f.write('multi_slot_desc {\n') ... f.write(' slots {\n') ... f.write(' name: "words"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write(' slots {\n') ... f.write(' name: "label"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write('}') >>> data_feed = base.DataFeedDesc('data.proto') >>> data_feed.set_batch_size(128) Args: batch_size (int): The number of batch size. Returns: None. N)r batch_size)rrs rset_batch_sizeDataFeedDesc.set_batch_size\sF&0"cURRS:wa [S5eUH8nSURRRUR UlM: g)a Set slots in :attr:`dense_slots_name` as dense slots. **Note: In default, all slots are sparse slots.** Features for a dense slot will be fed into a Tensor, while those for a sparse slot will be fed into a DenseTensor. Examples: .. code-block:: python >>> import paddle.base as base >>> with open("data.proto", "w") as f: ... f.write('name: "MultiSlotDataFeed"\n') ... f.write('batch_size: 2\n') ... f.write('multi_slot_desc {\n') ... f.write(' slots {\n') ... f.write(' name: "words"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write(' slots {\n') ... f.write(' name: "label"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write('}') >>> data_feed = base.DataFeedDesc('data.proto') >>> data_feed.set_dense_slots(['words']) Args: dense_slots_name (list(str)): a list of slot names which will be set dense. Returns: None. r zNOnly MultiSlotDataFeed needs set_dense_slots, please check your datafeed.protoTN)r r ValueErrorrrris_dense)rdense_slots_namers rset_dense_slotsDataFeedDesc.set_dense_slotssdL ??  #6 6` %D OO + + 1 1$$T* %r"cURRS:wa [S5eUH8nSURRRUR UlM: g)a Set if a specific slot will be used for training. A dataset shall contain a lot of features, through this function one can select which ones will be used for a specific model. Examples: .. code-block:: python >>> import paddle.base as base >>> with open("data.proto", "w") as f: ... f.write('name: "MultiSlotDataFeed"\n') ... f.write('batch_size: 2\n') ... f.write('multi_slot_desc {\n') ... f.write(' slots {\n') ... f.write(' name: "words"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write(' slots {\n') ... f.write(' name: "label"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write('}') >>> data_feed = base.DataFeedDesc('data.proto') >>> data_feed.set_use_slots(['words']) Args: use_slots_name: a list of slot names which will be used in training Note: Default is not used for all slots r zLOnly MultiSlotDataFeed needs set_use_slots, please check your datafeed.protoTN)r rr$rrris_used)ruse_slots_namers r set_use_slotsDataFeedDesc.set_use_slotssdH ??  #6 6^ #D OO + + 1 1$$T* #r"cB[R"UR5$)a Returns a protobuf message for this DataFeedDesc Examples: .. code-block:: python >>> import paddle.base as base >>> with open("data.proto", "w") as f: ... f.write('name: "MultiSlotDataFeed"\n') ... f.write('batch_size: 2\n') ... f.write('multi_slot_desc {\n') ... f.write(' slots {\n') ... f.write(' name: "words"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write(' slots {\n') ... f.write(' name: "label"\n') ... f.write(' type: "uint64"\n') ... f.write(' is_dense: false\n') ... f.write(' is_used: true\n') ... f.write(' }\n') ... f.write('}') >>> data_feed = base.DataFeedDesc('data.proto') >>> print(data_feed.desc()) Returns: A string message )rMessageToStringr )rs rdescDataFeedDesc.descs>**4??;;r")__name_to_indexr N) __name__ __module__ __qualname____firstlineno____doc__rr r'r,r0__static_attributes__r"rrrs$8t #0J-^+Zr=s(+ f<f