hdf5文件导入到DolphinDB database分布式表的问题

我的分布式数据库建表脚本如下:

login("admin","123456");
dfsPath = "dfs://bitmex"
if(existsDatabase(dfsPath)){
    dropDatabase(dfsPath)
}
db = database(dfsPath,RANGE, date(datetimeAdd(2018.01M,0..10*12,'M'))) 
modaltb = table(1:0, `open`high`low`close`volume`time,[DOUBLE,DOUBLE,DOUBLE,DOUBLE,DOUBLE,DATETIME])
db.createPartitionedTable(modaltb, `kline_eth_usd_kline,`time)

现在有一个hdf5文件,我想把它的数据导入到这个DolphinDB表中,但失败了。我的导入过程是这样的,先执行下面语句:

loadPlugin(getHomeDir()+"/plugins/hdf5/PluginHdf5.txt")
dataFilePath="/home/trudy/v1106/server/kline_eth_usd_kline_1m_clean_20180802_20200511.h5"
hdf5::lsTable(dataFilePath)

得到dataset;

tableName                tableDims        tableType
/df/_i_table/index/abounds    744    H5T_NATIVE_LLONG
/df/_i_table/index/bounds    247,3    H5T_NATIVE_LLONG
/df/_i_table/index/indices    253952,3    H5T_NATIVE_UINT
/df/_i_table/index/indicesLR    253952    H5T_NATIVE_UINT
/df/_i_table/index/mbounds    744    H5T_NATIVE_LLONG
/df/_i_table/index/mranges    3    H5T_NATIVE_LLONG
/df/_i_table/index/ranges    2,3    H5T_NATIVE_LLONG
/df/_i_table/index/sorted    253952,3    H5T_NATIVE_LLONG
/df/_i_table/index/sortedLR    254201    H5T_NATIVE_LLONG
/df/_i_table/index/zbounds    744    H5T_NATIVE_LLONG
/df/table    933170    H5T_COMPOUND

然后,调用loadHDF5Ex写入:

datasetName = "/df/table"
hdf5::loadHDF5Ex(db, "kline_eth_usd_kline", "time", dataFilePath,datasetName)

用下面语句查询,记录数为0,说明没写入成功。

dfsTable=loadTable(dfsPath,"kline_eth_usd_kline")
select count(*) from dfsTable

然后我用loadHDF5导入,:

hdf5::loadHDF5( dataFilePath,datasetName)

是可以看到有如下所示数据的:

index                        Aval0 Aval1 Aval2 Aval3 Aval4
1,533,200,820,000,000,000    400    400    400    400    1
1,533,200,880,000,000,000    420    420    420    400    1
1,533,200,940,000,000,000    420    420    420    420    0
...


请先 登录 后评论

1 个回答

Jason Tang - 时序数据库技术支持

hdf5文件中的时间戳数据是纳秒整数,分布式表是datetime,写入前需要进行转换。

t1=select Aval0,Aval1,Aval2,Aval3,Aval4,datetime(nanotimestamp(index)) as times from hdf5::loadHDF5( dataFilePath,datasetName) 
dfsTable=loadTable(dfsPath,"kline_eth_usd_kline")
dfsTable.append!(t1)


请先 登录 后评论