ai.djl.basicdataset

Class CsvDataset

java.lang.Object

ai.djl.training.dataset.RandomAccessDataset

ai.djl.basicdataset.CsvDataset

All Implemented Interfaces:

ai.djl.training.dataset.Dataset

Direct Known Subclasses:

AirfoilRandomAccess, AmazonReview, AmesRandomAccess

public class CsvDataset
extends ai.djl.training.dataset.RandomAccessDataset

CsvDataset represents the dataset that stored in a .csv file.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CsvDataset.CsvBuilder<T extends CsvDataset.CsvBuilder<T>> Used to build a CsvDataset.
static class	CsvDataset.Feature A class contains feature name and its Featurizer.
static interface	CsvDataset.Featurizer An interface that convert String to numeric data.

Nested classes/interfaces inherited from class ai.djl.training.dataset.RandomAccessDataset

ai.djl.training.dataset.RandomAccessDataset.BaseBuilder<T extends ai.djl.training.dataset.RandomAccessDataset.BaseBuilder<T>>

Nested classes/interfaces inherited from interface ai.djl.training.dataset.Dataset

ai.djl.training.dataset.Dataset.Usage

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.commons.csv.CSVFormat	csvFormat
protected java.util.List<org.apache.commons.csv.CSVRecord>	csvRecords
protected java.net.URL	csvUrl
protected java.util.List<CsvDataset.Feature>	features
protected java.util.List<CsvDataset.Feature>	labels

Fields inherited from class ai.djl.training.dataset.RandomAccessDataset

dataBatchifier, device, executor, labelBatchifier, limit, pipeline, prefetchNumber, sampler, targetPipeline

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	CsvDataset(CsvDataset.CsvBuilder<?> builder)

Method Summary

Modifier and Type	Method and Description
protected long	availableSize()
static CsvDataset.CsvBuilder<?>	builder() Creates a builder to build a AmesRandomAccess.
ai.djl.training.dataset.Record	get(ai.djl.ndarray.NDManager manager, long index)
java.util.List<java.lang.String>	getColumnNames() Returns the column names of the CSV file.
void	prepare(ai.djl.util.Progress progress)
protected ai.djl.ndarray.NDList	toNDList(ai.djl.ndarray.NDManager manager, org.apache.commons.csv.CSVRecord record, java.util.List<CsvDataset.Feature> selected)

Methods inherited from class ai.djl.training.dataset.RandomAccessDataset

getData, getData, randomSplit, size, subDataset, toArray

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ai.djl.training.dataset.Dataset

prepare

Field Detail

csvUrl

protected java.net.URL csvUrl

csvFormat

protected org.apache.commons.csv.CSVFormat csvFormat

features

protected java.util.List<CsvDataset.Feature> features

labels

protected java.util.List<CsvDataset.Feature> labels

csvRecords

protected java.util.List<org.apache.commons.csv.CSVRecord> csvRecords

Constructor Detail

CsvDataset

protected CsvDataset(CsvDataset.CsvBuilder<?> builder)

Method Detail

get

public ai.djl.training.dataset.Record get(ai.djl.ndarray.NDManager manager,
                                          long index)

Specified by:

get in class ai.djl.training.dataset.RandomAccessDataset

availableSize

protected long availableSize()

Specified by:

availableSize in class ai.djl.training.dataset.RandomAccessDataset

prepare

public void prepare(ai.djl.util.Progress progress)
             throws java.io.IOException

Throws:

java.io.IOException

builder

public static CsvDataset.CsvBuilder<?> builder()

Creates a builder to build a AmesRandomAccess.

Returns:

a new builder

getColumnNames

public java.util.List<java.lang.String> getColumnNames()

Returns the column names of the CSV file.

Returns:

a list of column name

toNDList

protected ai.djl.ndarray.NDList toNDList(ai.djl.ndarray.NDManager manager,
                                         org.apache.commons.csv.CSVRecord record,
                                         java.util.List<CsvDataset.Feature> selected)