利用DeepSeek辅助改写luadbi-duckdb支持日期和时间戳数据类型
1.克隆github上的luadbi存储库
root@66d4e20ec1d7:/par# git clone https://bgithub.xyz/mwild1/luadbi
Cloning into 'luadbi'...
remote: Enumerating objects: 1139, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 1139 (delta 9), reused 7 (delta 2), pack-reused 1099 (from 2)
Receiving objects: 100% (1139/1139), 265.55 KiB | 573.00 KiB/s, done.
Resolving deltas: 100% (732/732), done.
root@66d4e20ec1d7:/par# chmod 777 luadbi -R
root@66d4e20ec1d7:/par# cd luadbi
2.编译打包luadbi-duckdb rock包
root@66d4e20ec1d7:/par/luadbi# luarocks make luadbi-duckdb-scm-0.rockspec --pack-binary-rock DUCKDB_DIR=/par/141 DUCKDB_INCDIR=/par/141Missing dependencies for luadbi-duckdb scm-0:luadbi scm (not installed)luadbi-duckdb scm-0 depends on lua >= 5.1 (5.4-1 provided by VM: success)
luadbi-duckdb scm-0 depends on luadbi scm (not installed)
Warning: falling back to wget - install luasec >= 1.1 to get native HTTPS supportError: Could not satisfy dependency luadbi scm: No results matching query were found for Lua 5.4.
不成功,说缺少luadbi scm依赖,再编译打包luadbi-scm rock包并安装
root@66d4e20ec1d7:/par/luadbi# luarocks make luadbi-scm-0.rockspec --pack-binary-rock luadbi scm-0 depends on lua >= 5.1 (5.4-1 provided by VM: success)
No existing manifest. Attempting to rebuild...
luadbi scm-0 is now installed in /tmp/luarocks_luarocks-build-pack-luadbi-9734937 (license: MIT/X11)Packed: /par/luadbi/luadbi-scm-0.all.rockroot@66d4e20ec1d7:/par/luadbi# luarocks install luadbi-scm-0.all.rockluadbi scm-0 depends on lua >= 5.1 (5.4-1 provided by VM: success)
luadbi scm-0 is now installed in /usr/local (license: MIT/X11)Checking stability of dependencies in the absence of
luadbi 0.7.4-1...Will not remove luadbi 0.7.4-1.
Removing it would break dependencies for:
luadbi-duckdb 0.7.4-1
luadbi-sqlite3 0.7.4-1Use --force to force removal (warning: this may break modules).
再次编译打包luadbi-duckdb rock包,仍然不行,报错和刚才一样
编辑luadbi-duckdb-scm-0.rockspec,将依赖部分注释掉
-- dependencies = {
-- "lua >= 5.1",
-- "luadbi = scm"
-- }
编译打包luadbi-duckdb rock包就通过了。
root@66d4e20ec1d7:/par/luadbi# luarocks make luadbi-duckdb-scm-0.rockspec --pack-binary-rock DUCKDB_DIR=/par/141 DUCKDB_INCDIR=/par/141gcc -O2 -fPIC -I/usr/local/include -c dbd/common.c -o dbd/common.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/main.c -o dbd/duckdb/main.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/statement.c -o dbd/duckdb/statement.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/connection.c -o dbd/duckdb/connection.o -I/par/141 -I./
gcc -shared -o /tmp/luarocks_build-luadbi-duckdb-scm-0-5683361/dbd/duckdb.so dbd/common.o dbd/duckdb/main.o dbd/duckdb/statement.o dbd/duckdb/connection.o -L/par/141/lib/aarch64-linux-gnu -Wl,-rpath,/par/141/lib/aarch64-linux-gnu -lduckdb
No existing manifest. Attempting to rebuild...
luadbi-duckdb scm-0 is now installed in /tmp/luarocks_luarocks-build-pack-luadbi-duckdb-8434912 (license: MIT/X11)Packed: /par/luadbi/luadbi-duckdb-scm-0.linux-aarch64.rock
3.安装luadbi-duckdb rock包,luarocks自动删除了原有的luadbi-duckdb 0.7.4-1版本。
root@66d4e20ec1d7:/par/luadbi# luarocks install luadbi-duckdb-scm-0.linux-aarch64.rockluadbi-duckdb scm-0 is now installed in /usr/local (license: MIT/X11)Checking stability of dependencies in the absence of
luadbi-duckdb 0.7.4-1...Removing luadbi-duckdb 0.7.4-1...
Removal successful.
4.试运行带有日期类型的表
lua duck.luaSegmentation fault (core dumped)
恢复安装luadbi-duckdb-0.7.4-1版本
root@66d4e20ec1d7:/par/luarocks-3.12.2# luarocks install luadbi-duckdb-0.7.4-1.src.rock DUCKDB_DIR=/par/141 DUCKDB_INCDIR=/par/141Missing dependencies for luadbi-duckdb 0.7.4-1:luadbi 0.7.4 (not installed)luadbi-duckdb 0.7.4-1 depends on lua >= 5.1, < 5.5 (5.4-1 provided by VM: success)
luadbi-duckdb 0.7.4-1 depends on luadbi 0.7.4 (not installed)
Warning: falling back to wget - install luasec >= 1.1 to get native HTTPS support
Installing https://luarocks.org/luadbi-0.7.4-1.src.rockluadbi 0.7.4-1 depends on lua >= 5.1, < 5.5 (5.4-1 provided by VM: success)
luadbi 0.7.4-1 is now installed in /usr/local (license: MIT/X11)Checking stability of dependencies in the absence of
luadbi scm-0...Removing luadbi scm-0...
Removal successful.
gcc -O2 -fPIC -I/usr/local/include -c dbd/common.c -o dbd/common.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/main.c -o dbd/duckdb/main.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/statement.c -o dbd/duckdb/statement.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/connection.c -o dbd/duckdb/connection.o -I/par/141 -I./
gcc -shared -o /tmp/luarocks_build-luadbi-duckdb-0.7.4-1-5600173/dbd/duckdb.so dbd/common.o dbd/duckdb/main.o dbd/duckdb/statement.o dbd/duckdb/connection.o -L/par/141/lib/aarch64-linux-gnu -Wl,-rpath,/par/141/lib/aarch64-linux-gnu -lduckdb
luadbi-duckdb 0.7.4-1 is now installed in /usr/local (license: MIT/X11)root@66d4e20ec1d7:/par/luarocks-3.12.2# cd /par
root@66d4e20ec1d7:/par# lua duck.lualua: duck.lua:16: Execute failed unknown datatype to bind
stack traceback:[C]: in for iterator 'for iterator'duck.lua:16: in main chunk[C]: in ?
还是unknown datatype to bind错误,而0.7.4-1版本statement.c的statement_fetch_impl函数相应代码是
case DUCKDB_TYPE_BLOB:case DUCKDB_TYPE_VARCHAR: {duckdb_string_t *vector_data = (duckdb_string_t *) duckdb_vector_get_data(col1);duckdb_string_t str = vector_data[ statement->cur_row ];if (named_columns) {LDB_PUSH_ATTRIB_STRING( str );} else {LDB_PUSH_ARRAY_STRING(i + 1, str);}}break;default:luaL_error(L, DBI_ERR_EXECUTE_FAILED, "unknown datatype to bind");break;
这样处理更安全。
为什么新版不报错直接崩溃?打开新版的源代码,原来它被修改成了:
default:case DUCKDB_TYPE_BLOB:case DUCKDB_TYPE_VARCHAR: {duckdb_string_t *vector_data = (duckdb_string_t *) duckdb_vector_get_data(col1);duckdb_string_t str = vector_data[ statement->cur_row ];if (named_columns) {LDB_PUSH_ATTRIB_STRING( str );} else {LDB_PUSH_ARRAY_STRING(i + 1, str);}}break;
未处理的数据类型都一律当做字符串输出,而它们的数据结构不是字符串,于是引发了崩溃。
解决的办法是根据相应数据类型来处理。从duckdb.h头文件找到相应的类型结构定义和转换函数原型,对DUCKDB_TYPE_DATE的处理如下:
case DUCKDB_TYPE_DATE:{
int32_t *res = duckdb_vector_get_data(col1);
duckdb_date date;
date.days=*res;
duckdb_date_struct dst=duckdb_from_date(date);
char a[100];
sprintf(a,"%04d-%02d-%02d",dst.year,dst.month,dst.day);
duckdb_string_t str= convert_to_duckdb_string(a);if (named_columns) {LDB_PUSH_ATTRIB_STRING( str );} else {LDB_PUSH_ARRAY_STRING(i + 1, str);}}break;
其中前半部分复制自DUCKDB_TYPE_INTEGER类型处理,后半部分复制自DUCKDB_TYPE_VARCHAR类型处理,
把c字符串转为duckdb_string_t的代码交给DeepSeek编写如下,放到上述函数定义的前面。
// 将C字符串转换为duckdb_string_t结构
duckdb_string_t convert_to_duckdb_string(const char* str) {duckdb_string_t result;if (str == NULL) {// 处理空指针情况result.value.inlined.length = 0;memset(result.value.inlined.inlined, 0, 12);return result;}size_t len = strlen(str);if (len <= 12) {// 使用内联存储result.value.inlined.length = (uint32_t)len;if (len > 0) {memcpy(result.value.inlined.inlined, str, len);}// 填充剩余空间为0if (len < 12) {memset(result.value.inlined.inlined + len, 0, 12 - len);}} else {// 使用指针存储result.value.pointer.length = (uint32_t)len;// 复制前4个字符到prefixmemcpy(result.value.pointer.prefix, str, 4);// 分配内存并复制完整字符串result.value.pointer.ptr = (char*)malloc(len + 1);if (result.value.pointer.ptr != NULL) {memcpy(result.value.pointer.ptr, str, len);result.value.pointer.ptr[len] = '\0';} else {// 内存分配失败,回退到内联存储(截断)result.value.inlined.length = 12;memcpy(result.value.inlined.inlined, str, 12);}}return result;
}
保存修改后重新编译打包luadbi-duckdb-scm,
root@66d4e20ec1d7:/par/luadbi# luarocks make luadbi-duckdb-scm-0.rockspec --pack-binary-rock DUCKDB_DIR=/par/141 DUCKDB_INCDIR=/par/141gcc -O2 -fPIC -I/usr/local/include -c dbd/common.c -o dbd/common.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/main.c -o dbd/duckdb/main.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/statement.c -o dbd/duckdb/statement.o -I/par/141 -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/connection.c -o dbd/duckdb/connection.o -I/par/141 -I./
gcc -shared -o /tmp/luarocks_build-luadbi-duckdb-scm-0-4883179/dbd/duckdb.so dbd/common.o dbd/duckdb/main.o dbd/duckdb/statement.o dbd/duckdb/connection.o -L/par/141/lib/aarch64-linux-gnu -Wl,-rpath,/par/141/lib/aarch64-linux-gnu -lduckdb
No existing manifest. Attempting to rebuild...
luadbi-duckdb scm-0 is now installed in /tmp/luarocks_luarocks-build-pack-luadbi-duckdb-9673602 (license: MIT/X11)Packed: /par/luadbi/luadbi-duckdb-scm-0.linux-aarch64.rockroot@66d4e20ec1d7:/par/luadbi# luarocks install luadbi-duckdb-scm-0.linux-aarch64.rock --forceluadbi-duckdb scm-0 is now installed in /usr/local (license: MIT/X11)root@66d4e20ec1d7:/par/luadbi# lua /par/duck.lua15
abc
张先生
2025-09-22
可见成功输出了日期类型,那时间戳类型只要一样照抄,也交给DeepSeek完成,相应的处理代码
case DUCKDB_TYPE_TIMESTAMP:{
int64_t *res = duckdb_vector_get_data(col1);
duckdb_timestamp date;
date.micros=*res;
duckdb_timestamp_struct dts=duckdb_from_timestamp(date);
char a[100];
sprintf(a,"%04d-%02d-%02d %02d:%02d:%02d.%06d",dts.date.year, dts.date.month, dts.date.day,dts.time.hour, dts.time.min, dts.time.sec,dts.time.micros);
duckdb_string_t str= convert_to_duckdb_string(a);if (named_columns) {LDB_PUSH_ATTRIB_STRING( str );} else {LDB_PUSH_ARRAY_STRING(i + 1, str);}}break;
保存后重新打包安装,时间戳类型也处理好了,那个default分支还是单独拿出来比较好,至少不会内存崩溃。
root@66d4e20ec1d7:/par/luadbi# lua /par/duck.lua15
2025-09-25 12:26:55.123456
张先生
2025-09-22
最终用于测试的lua脚本如下
#!/usr/bin/env lua
DBI = require "DBI"dbd, err = DBI.Connect( 'DuckDB', 'lua_duckdb', 'dbuser', 'password' )
assert(dbd, err)dbd:autocommit(true)
statement = dbd:prepare( "create or replace table table_1 as select 15 id, timestamp'2025-09-25 12:26:55.123456' column1,'张先生' column2,date'2025-09-22' column3 ;" )
statement:execute()statement = assert(dbd:prepare( "select * from table_1 where id = $1;" ))
statement:execute( 15 )for row in statement:rows(true) doprint(row['id'])print(row['column1'])print(row['column2'])print(row['column3'])end
