用Lua访问DuckDB数据库
首先从源代码安装Lua
wget https://www.lua.org/ftp/lua-5.4.8.tar.gz -q
tar xf lua-5.4.8.tar.gz
cd  lua-5.4.8
make all test
make install
然后从源代码安装包管理工具luarocks
wget https://github.com/luarocks/luarocks/releases/download/v3.12.2/luarocks-3.12.2.tar.gz -q
tar xf luarocks-3.12.2.tar.gz
cd luarocks-3.12.2
./configure && make && make install
安装luasocket包
luarocks install luasocket
如果失败,可以先下载包,再找到文件名,再安装文件,如下所示
luarocks download luasocket
ls luasocket*
luasocket-3.1.0-1.src.rock
luarocks install luasocket-3.1.0-1.src.rock
安装DuckDB数据库访问接口包
luarocks download luadbi-duckdb
Warning: falling back to wget - install luasec >= 1.1 to get native HTTPS support
luarocks install luadbi-duckdb-0.7.4-1.src.rockError: Could not find header file for DUCKDBNo file duckdb.h in /usr/local/includeNo file duckdb.h in /usr/includeNo file duckdb.h in /include
You may have to install DUCKDB in your system and/or pass DUCKDB_DIR or DUCKDB_INCDIR to the luarocks command.
Example: luarocks install luadbi-duckdb DUCKDB_DIR=/usr/local
提示需要设置DUCKDB目录,我的libduckdb目录保存了头文件和库文件,所以如下编写
luarocks install luadbi-duckdb-0.7.4-1.src.rock DUCKDB_DIR=/par/libduckdb DUCKDB_INCDIR=/par/libduckdbMissing dependencies for luadbi-duckdb 0.7.4-1:luadbi 0.7.4 (not installed)luadbi-duckdb 0.7.4-1 depends on lua >= 5.1, < 5.5 (5.4-1 provided by VM: success)
luadbi-duckdb 0.7.4-1 depends on luadbi 0.7.4 (not installed)
Warning: falling back to wget - install luasec >= 1.1 to get native HTTPS support
Installing https://luarocks.org/luadbi-0.7.4-1.src.rockluadbi 0.7.4-1 depends on lua >= 5.1, < 5.5 (5.4-1 provided by VM: success)
luadbi 0.7.4-1 is now installed in /usr/local (license: MIT/X11)gcc -O2 -fPIC -I/usr/local/include -c dbd/common.c -o dbd/common.o -I/par/libduckdb -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/main.c -o dbd/duckdb/main.o -I/par/libduckdb -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/statement.c -o dbd/duckdb/statement.o -I/par/libduckdb -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/connection.c -o dbd/duckdb/connection.o -I/par/libduckdb -I./
gcc  -shared -o /tmp/luarocks_build-luadbi-duckdb-0.7.4-1-4954134/dbd/duckdb.so dbd/common.o dbd/duckdb/main.o dbd/duckdb/statement.o dbd/duckdb/connection.o -L/par/libduckdb/lib/x86_64-linux-gnu -Wl,-rpath,/par/libduckdb/lib/x86_64-linux-gnu -lduckdb
/usr/bin/ld: cannot find -lduckdb: No such file or directory
collect2: error: ld returned 1 exit statusError: Build error: Failed compiling module dbd/duckdb.so
先自动安装了依赖项luadbi,编译luadbi-duckdb完成,链接失败,还需要设置LIBRARY_PATH和LD_LIBRARY_PATH环境变量。
export LIBRARY_PATH=$LIBRARY_PATH:/par/libduckdb
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/par/libduckdb
luarocks install luadbi-duckdb-0.7.4-1.src.rock DUCKDB_DIR=/par/libduckdb DUCKDB_INCDIR=/par/libduckdbluadbi-duckdb 0.7.4-1 depends on lua >= 5.1, < 5.5 (5.4-1 provided by VM: success)
luadbi-duckdb 0.7.4-1 depends on luadbi 0.7.4 (0.7.4-1 installed: success)
gcc -O2 -fPIC -I/usr/local/include -c dbd/common.c -o dbd/common.o -I/par/libduckdb -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/main.c -o dbd/duckdb/main.o -I/par/libduckdb -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/statement.c -o dbd/duckdb/statement.o -I/par/libduckdb -I./
gcc -O2 -fPIC -I/usr/local/include -c dbd/duckdb/connection.c -o dbd/duckdb/connection.o -I/par/libduckdb -I./
gcc  -shared -o /tmp/luarocks_build-luadbi-duckdb-0.7.4-1-7237304/dbd/duckdb.so dbd/common.o dbd/duckdb/main.o dbd/duckdb/statement.o dbd/duckdb/connection.o -L/par/libduckdb/lib/x86_64-linux-gnu -Wl,-rpath,/par/libduckdb/lib/x86_64-linux-gnu -lduckdb
luadbi-duckdb 0.7.4-1 is now installed in /usr/local (license: MIT/X11)
项目没有DuckDB的例子,发版日志中说New driver for DuckDB. Largely untested and should be considered expiremental at this time.
从wiki中找了PostgreSQL的例子
#!/usr/bin/env lua
DBI = require "DBI"dbd, err = DBI.Connect( 'PostgreSQL', 'your_database', 'dbuser', 'password' )
assert(dbd, err)dbd:autocommit(true)statement = dbd:prepare( "select * from table_1 where id = $1;" )
statement:execute( 15 )for row in statement:rows(true) doprint(row['column1'])print(row['column2'])print(row['column3'])end
修改为
#!/usr/bin/env lua
DBI = require "DBI"dbd, err = DBI.Connect( 'DuckDB', 'your_database', 'dbuser', 'password' )
assert(dbd, err)dbd:autocommit(true)
-- statement = dbd:prepare( "create table table_1 as select 15 id, 'abc' column1,'张先生' column2,date'2025-09-22' column3 ;" )
-- statement:execute()
statement = dbd:prepare( "select * from table_1 where id = $1;" )
statement:execute( 15 )for row in statement:rows(true) do
print(row['id'])--print(row['column1'])--print(row['column2'])-- print(row['column3'])end
因为没有现成的DuckDB数据库,所以用create table语句先创建表,执行成功
lua duck.lua
lua: duck.lua:13: Execute failed unknown datatype to bind
stack traceback:[C]: in for iterator 'for iterator'duck.lua:13: in main chunk[C]: in ?./duckdb141 your_database
DuckDB v1.4.1 (Andium) b390a7c376
Enter ".help" for usage hints.
D from table_1;
┌───────┬─────────┬─────────┬────────────┐
│  id   │ column1 │ column2 │  column3   │
│ int32 │ varchar │ varchar │    date    │
├───────┼─────────┼─────────┼────────────┤
│  15   │ abc     │ 张先生  │ 2025-09-22 │
└───────┴─────────┴─────────┴────────────┘
但是后面的select查询一直失败,直到最简单的print(row[‘id’])都是上述错误。
继续用luadbi-duckdb自带的测试程序测试
root@6ae32a5ffcde:/par/luadbi-0.7.5# cd tests
root@6ae32a5ffcde:/par/luadbi-0.7.5/tests# lua run_tests.lua
lua: run_tests.lua:7: module 'busted.runner' not found:No LuaRocks module found for busted.runnerno field package.preload['busted.runner']no file '../busted/runner.lua'no file '/usr/local/share/lua/5.4/busted/runner.lua'no file '/usr/local/share/lua/5.4/busted/runner/init.lua'no file '/usr/local/lib/lua/5.4/busted/runner.lua'no file '/usr/local/lib/lua/5.4/busted/runner/init.lua'no file './busted/runner.lua'no file './busted/runner/init.lua'no file '/root/.luarocks/share/lua/5.4/busted/runner.lua'no file '/root/.luarocks/share/lua/5.4/busted/runner/init.lua'no file '../busted/runner.so'no file '/usr/local/lib/lua/5.4/busted/runner.so'no file '/usr/local/lib/lua/5.4/loadall.so'no file './busted/runner.so'no file '/root/.luarocks/lib/lua/5.4/busted/runner.so'no file '../busted.so'no file '/usr/local/lib/lua/5.4/busted.so'no file '/usr/local/lib/lua/5.4/loadall.so'no file './busted.so'no file '/root/.luarocks/lib/lua/5.4/busted.so'
stack traceback:[C]: in function 'require'run_tests.lua:7: in main chunk[C]: in ?
root@6ae32a5ffcde:/par/luadbi-0.7.5/tests#
提示需要安装busted包。
luarocks install busted
Warning: falling back to wget - install luasec >= 1.1 to get native HTTPS support
Installing https://luarocks.org/busted-2.2.0-1.src.rockMissing dependencies for busted 2.2.0-1:lua_cliargs 3.0 (not installed)luasystem >= 0.2.0 (not installed)dkjson >= 2.1.0 (not installed)say >= 1.4-1 (not installed)luassert >= 1.9.0-1 (not installed)lua-term >= 0.1 (not installed)penlight >= 1.3.2 (not installed)mediator_lua >= 1.1.1 (not installed)
..
busted 2.2.0-1 depends on mediator_lua >= 1.1.1 (1.1.2-0 installed: success)
busted 2.2.0-1 is now installed in /usr/local (license: MIT <http://opensource.org/licenses/MIT>)root@6ae32a5ffcde:/par/luadbi-0.7.5# cd tests
root@6ae32a5ffcde:/par/luadbi-0.7.5/tests# lua run_tests.lua
●●◼◼◼◼◼◼◼◼●●
4 successes / 8 failures / 3 errors / 0 pending : 1.556044 secondsFailure → run_tests.lua @ 745
DuckDB #duckdb Tests simple selects
run_tests.lua:130: Expected objects to be the same.
Passed in:
(string) 'Error preparing statement handle: Catalog Error: Table with name select_tests does not exist!root@6ae32a5ffcde:/par/luadbi-0.7.5/tests# /par/duckdb141 duckdb-test
DuckDB v1.4.1 (Andium) b390a7c376
Enter ".help" for usage hints.
D .read schemas/duckdb.sql
D show tables;
┌──────────────┐
│     name     │
│   varchar    │
├──────────────┤
│ insert_tests │
│ select_tests │
│ update_tests │
└──────────────┘
D .exit
root@6ae32a5ffcde:/par/luadbi-0.7.5/tests# lua run_tests.lua
●●●●●●●●●●●●
12 successes / 0 failures / 3 errors / 0 pending : 1.19215 secondsError → run_tests.lua @ 675
PostgreSQL #psql setup
../DBI.lua:54: Cannot load driver PostgreSQL. Available drivers are: DuckDB
安装后再测试,提示缺少表,用schemas/duckdb.sql创建表,DuckDB相关的测试都通过了,经过比较发现,我的程序使用了日期类型,而在luadbi-duckdb 源代码中只支持整数浮点布尔和字符串,其他类型默认, Lua 没有内置的日期时间类型,需要增加转换的代码,它还没有完成。
default:luaL_error(L, DBI_ERR_EXECUTE_FAILED, "unknown datatype to bind");break;
通过把日期型改为字符串,执行结果如下
lua duck.lua
15
abc
张先生
2025-09-22
libduckdb.so不在搜索路径的问题可以通过软链接ln -s /par/141/libduckdb.so /usr/lib/aarch64-linux-gnu/libduckdb.so解决。这样就不需要设置LD_LIBRARY_PATH环境变量了。
