插入pandas数据问题 RuntimeError: The value 342832506232.02000000 (column "total_base", row 179) must be of DECIMAL64 type.


share streamTable(1000:0, `Symbol`Datetime`ask`bid`price`total_base`total_quote,[SYMBOL,DATETIME,DECIMAL64(8),DECIMAL64(8),DECIMAL64(8),DECIMAL64(8),DECIMAL64(8)]) as t24
ds=[{'s': 's0', 'C': 1702626717012, 'a': '2.16000000', 'b': '2.15900000', 'c': '2.16000000', 'v': '31405307.17000000', 'q': '70186955.16226000'},
 {'s': 's1', 'C': 1702626716568, 'a': '1.96700000', 'b': '1.96500000', 'c': '1.97400000', 'v': '24582.80000000', 'q': '49790.64185000'},
 {'s': 's2', 'C': 1702626716392, 'a': '0.00137900', 'b': '0.00137800', 'c': '0.00137800', 'v': '2572194212.00000000', 'q': '3569301.93842800'},
 {'s': 's3', 'C': 1702626716168, 'a': '0.00095300', 'b': '0.00095100', 'c': '0.00095100', 'v': '439505.91000000', 'q': '430.04806597'},
 {'s': 's4', 'C': 1702626716050, 'a': '0.53130000', 'b': '0.53120000', 'c': '0.53120000', 'v': '18374212.70000000', 'q': '10000475.73016000'},
 {'s': 's5', 'C': 1702626716780, 'a': '0.00017714', 'b': '0.00017713', 'c': '0.00017714', 'v': '348378610661.95000000', 'q': '63049321.62639020'},
 {'s': 's6', 'C': 1702626716759, 'a': '0.00113500', 'b': '0.00113300', 'c': '0.00113300', 'v': '2371.65800000', 'q': '2.71165406'}]
In [155]: from decimal import Decimal                                
     ...: df = pandas.DataFrame(ds)     
     ...: dt = pandas.DataFrame()       
     ...: dt['Symbol']  =df['s']        
     ...: dt['Datetime']=pandas.to_datetime(df['C'],unit='ms')                  
     ...: dt['ask']     =df['a'].apply(lambda x: Decimal(x))                    
     ...: dt['bid']     =df['b'].apply(lambda x: Decimal(x))                    
     ...: dt['price']   =df['c'].apply(lambda x: Decimal(x))                    
     ...: dt['total_base']=df['v'].apply(lambda x: Decimal(x))                  
     ...: dt['total_quote']=df['q'].apply(lambda x: Decimal(x))                 
     ...:                               
     ...: print(U.len(ds,df,dt))        
     ...: s.upload({"dt":dt})           
     ...: s.run("tableInsert(t24,dt)")  
     ...:   
  ---> 14 s.upload({"dt":dt})
     15 s.run("tableInsert(t24,dt)")

~/anaconda3/lib/python3.9/site-packages/dolphindb/session.py in upload(self, nameObjectDict)
    446             A pandas DataFrame corresponds to DolphinDB table.
    447         """
--> 448         return self.cpp.upload(nameObjectDict)
    449 
    450     def run(self, script: str, *args, **kwargs):

RuntimeError: The value 348378610661.95000000 (column "total_base", row 5) must be of DECIMAL64 type.           


python数据源插入流表计算,  通过pandas转换后再调用 s.upload({"dt":dt}); s.run("tableInsert(t24,dt)") 是最高效的方法吗?dolphinDB直接把json数据转换为表会不会快一点



DECIMAL128  还是出错   share streamTable(1000:0, `Symbol`Datetime`ask`bid`price`total_base`total_quote,[SYMBOL,DATETIME,DECIMAL128(8),DECIMAL128(8),DECIMAL128(8),DECIMAL128(8),DECIMAL128(8)]) as t24
---> 14 s.upload({"dt":dt})
     15 s.run("tableInsert(t24,dt)")

~/anaconda3/lib/python3.9/site-packages/dolphindb/session.py in upload(self, nameObjectDict)
    446             A pandas DataFrame corresponds to DolphinDB table.
    447         """
--> 448         return self.cpp.upload(nameObjectDict)
    449 
    450     def run(self, script: str, *args, **kwargs):

RuntimeError: The value 361791112827.48000000 (column "total_base", row 204) must be of DECIMAL64 type.
就算插入成功,python读取时也出错:
In [11]: s.run('select * from t24')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-aedc0d7f49cc> in <module>
----> 1 s.run('select * from t24')

~/anaconda3/lib/python3.9/site-packages/dolphindb/session.py in run(self, script, *args, **kwargs)
    475             if "fetchSize" in kwargs.keys():
    476                 return BlockReader(self.cpp.runBlock(script, **kwargs))
--> 477         return self.cpp.run(script, *args, **kwargs)
    478 
    479     def runFile(self, filepath: str, *args, **kwargs):

RuntimeError: <Exception> in run: Not allowed to create a vector with type DECIMAL128
dolphindb.__version__ '1.30.22.6'升级版本后,可以读取了,但是值不对啊 ,原始值 258585503928.37000000 表中读出 74118063191.27448384
In [3]: df = pandas.DataFrame(ds)   ...: dt = pandas.DataFrame()   ...: dt['Symbol']  =df['s']
   ...: dt['Datetime']=pandas.to_datetime(df['C'],unit='ms')   ...: dt['ask']     =df['a'].apply(lambda x: Decimal(x))
   ...: dt['bid']     =df['b'].apply(lambda x: Decimal(x))
   ...: dt['price']   =df['c'].apply(lambda x: Decimal(x))
   ...: dt['total_base']=df['v'].apply(lambda x: Decimal(x))
   ...: dt['total_quote']=df['q'].apply(lambda x: Decimal(x))
   ...: 
   ...: print(U.len(ds,df,dt))
   ...: s.upload({"dt":dt})
   ...: s.run("tableInsert(t24,dt)")
[1, 1, 1]
Out[3]: 1

In [4]: s.run('select * from t24')
Out[4]: 
     Symbol            Datetime         ask         bid       price            total_base        total_quote
0  symbol22 2023-12-16 06:40:31  0.00016719  0.00016718  0.00016721  74118063191.27448384  44395856.11508572

In [5]: dt
Out[5]: 
     Symbol                Datetime         ask         bid       price             total_base        total_quote
0  symbol22 2023-12-16 06:40:31.584  0.00016719  0.00016718  0.00016721  258585503928.37000000  44395856.11508572


请先 登录 后评论

1 个回答

wale

1.30.22.6 版本 PROTOCOL_DDB 协议新增支持数据类型 DECIMAL128。你用PROTOCOL_DDB 协议试试

请先 登录 后评论