hdf5文件中的时间戳数据是纳秒整数,分布式表是datetime,写入前需要进行转换。
t1=select Aval0,Aval1,Aval2,Aval3,Aval4,datetime(nanotimestamp(index)) as times from hdf5::loadHDF5( dataFilePath,datasetName) dfsTable=loadTable(dfsPath,"kline_eth_usd_kline") dfsTable.append!(t1)
我的分布式数据库建表脚本如下:
login("admin","123456"); dfsPath = "dfs://bitmex" if(existsDatabase(dfsPath)){ dropDatabase(dfsPath) } db = database(dfsPath,RANGE, date(datetimeAdd(2018.01M,0..10*12,'M'))) modaltb = table(1:0, `open`high`low`close`volume`time,[DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DATETIME]) db.createPartitionedTable(modaltb, `kline_eth_usd_kline,`time)
现在有一个hdf5文件,我想把它的数据导入到这个DolphinDB表中,但失败了。我的导入过程是这样的,先执行下面语句:
loadPlugin(getHomeDir()+"/plugins/hdf5/PluginHdf5.txt") dataFilePath="/home/trudy/v1106/server/kline_eth_usd_kline_1m_clean_20180802_20200511.h5" hdf5::lsTable(dataFilePath)
得到dataset;
tableName tableDims tableType /df/_i_table/index/abounds 744 H5T_NATIVE_LLONG /df/_i_table/index/bounds 247,3 H5T_NATIVE_LLONG /df/_i_table/index/indices 253952,3 H5T_NATIVE_UINT /df/_i_table/index/indicesLR 253952 H5T_NATIVE_UINT /df/_i_table/index/mbounds 744 H5T_NATIVE_LLONG /df/_i_table/index/mranges 3 H5T_NATIVE_LLONG /df/_i_table/index/ranges 2,3 H5T_NATIVE_LLONG /df/_i_table/index/sorted 253952,3 H5T_NATIVE_LLONG /df/_i_table/index/sortedLR 254201 H5T_NATIVE_LLONG /df/_i_table/index/zbounds 744 H5T_NATIVE_LLONG /df/table 933170 H5T_COMPOUND
然后,调用loadHDF5Ex写入:
datasetName = "/df/table" hdf5::loadHDF5Ex(db, "kline_eth_usd_kline", "time", dataFilePath,datasetName)
用下面语句查询,记录数为0,说明没写入成功。
dfsTable=loadTable(dfsPath,"kline_eth_usd_kline") select count(*) from dfsTable
然后我用loadHDF5导入,:
hdf5::loadHDF5( dataFilePath,datasetName)
是可以看到有如下所示数据的:
index Aval0 Aval1 Aval2 Aval3 Aval4 1,533,200,820,000,000,000 400 400 400 400 1 1,533,200,880,000,000,000 420 420 420 400 1 1,533,200,940,000,000,000 420 420 420 420 0 ...
hdf5文件中的时间戳数据是纳秒整数,分布式表是datetime,写入前需要进行转换。
t1=select Aval0,Aval1,Aval2,Aval3,Aval4,datetime(nanotimestamp(index)) as times from hdf5::loadHDF5( dataFilePath,datasetName) dfsTable=loadTable(dfsPath,"kline_eth_usd_kline") dfsTable.append!(t1)