ddb查询之后资源无法释放

您好,在Linux服务器上部署了ddb2.00.7版本的服务器,然后建表建库存储股票K线数据。建库建表的代码如下:

def createDB(dbName, tbName1, tbName2, tbName3) {
	if (existsDatabase(dbName))
		dropDatabase(dbName)
	
	dbDate = 1990.01M..2050.12M
	db = database(dbName, VALUE, dbDate, , "TSDB")
	
	cols = `date`code`exchange`open`high`low`close`preclose`volume`turnover`turnrate`tradestatus`pctChg`peTTM`pbMRQ`psTTM`pcfNcfTTM`isST
	type=`DATE`STRING`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`SYMBOL
	tb = table(1:0, cols, type)
	db.createPartitionedTable(tb, tbName1, `date, sortColumns=`code`date, keepDuplicates=FIRST)

	cols1 = `datetime`code`exchange`open`high`low`close`volume`turnover
	type1 =`TIMESTAMP`STRING`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE
	tb = table(1:0, cols1, type1)
	db.createPartitionedTable(tb, tbName2,  `datetime, sortColumns=`code`datetime, keepDuplicates=FIRST)

	cols2 = `datetime`code`exchange`open`high`low`close`volume`turnover
	type2 =`TIMESTAMP`STRING`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE
	tb = table(1:0, cols2, type2)
	db.createPartitionedTable(tb, tbName3, `datetime, sortColumns=`code`datetime, keepDuplicates=FIRST)
}


但是上传数据之后每次查询大一点的规模,内存占用就会堆积,资源无法释放掉。上传数据可以正常释放内存资源,只有查询才会出现这种问题。重新部署服务器之后还是有这个问题。服务器状态显示如下:

内存已用 (已分配) / 最大可用CPU 用量 (平均负载)当前连接 | 最大连接硬盘读取 | 硬盘写入网络接收 | 网络发送排队作业 | 运行作业排队任务 | 运行任务Workers3.7 GB (3.8 GB) / 8 GB1.6% (0.0)2 | 5122.8 KB/s | 0 B/s363.4 KB/s | 12.1 KB/s0 | 00 | 04 | 3 | 0


日志截取如下:



2022-08-18 02:18:09.160141 <INFO> :cleanExpiredTransactionsOnMaster starts
2022-08-18 02:18:01.128744 <INFO> :chunkNode clean closed transactions successfully.
2022-08-18 02:18:01.128712 <INFO> :IoTransactionMgmt::collectGarbage execute successfully.
2022-08-18 02:17:52.127134 <INFO> :RedologMgmt garbage collect finish and engine is OLAP
2022-08-18 02:17:51.126914 <INFO> :RedologMgmt garbage collect finish and engine is OLAP
2022-08-18 02:17:49.156571 <INFO> :cleanExpiredTransactionsOnMaster starts
2022-08-18 02:17:44.125712 <INFO> :chunksCollectGarbage successfully.
2022-08-18 02:17:39.124885 <INFO> :no gc candidates, skip garbage collection
2022-08-18 02:17:39.124841 <INFO> :start cleaning garbage versions
2022-08-18 02:17:30.123311 <INFO> :<ChunkMgmt::createVersionGc> ignore this time because NEED_CHECK_CREATE_VERSION flag is false.


我已经调用了如下命令试图清理缓存数据,但是没有起作用。

flushTSDBCache();

undef all;
clearAllCache();


服务器配置如下:

mode=single
maxMemSize=32
maxConnections=512
workerNum=4
localExecutors=3
dataSync=1
chunkCacheEngineMemSize=16
maxLogSize=256
memoryReleaseRate=10
warningMemSize=6
newValuePartitionPolicy=add
maxPubConnections=64
subExecutors=4
subPort=8849
lanCluster=0


查询就是很常见的查询,比如按日期、按股票ID等查询,没有特别的计算操作,只是简单查询满足某些条件的数据返回。

这个问题如果不解决数据库都没法用,多查询几次就爆内存了。希望能够获取到帮助,之前群里请教过,有技术人员帮我远程操作了一下,但是没有给出什么有效的解决方案。万分感谢~~

请先 登录 后评论

1 个回答

Bob

最大内存只有8G?和MemSize有关的配置相应调整到8G以下试试,参考内存配置参数

请先 登录 后评论