当前位置：首页 > news >正文

langchain4j+local-ai小试牛刀

news 来源：原创 2025/6/21 2:39:43

序

本文主要研究一下如何本地运行local-ai并通过langchain4j集成调用。

步骤

curl安装

curl https://localai.io/install.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21509    0 21509    0     0  45959      0 --:--:-- --:--:-- --:--:-- 45861
  Downloading local-ai...
######################################################################## 100.0%
  Installing local-ai to /usr/local/bin...
  Creating local-ai user...
  Adding current user to local-ai group...
  Creating local-ai systemd service...
  The LocalAI API is now available at 127.0.0.1:8080.
  Install complete. Run "local-ai" from the command line.
[WARN]  No NVIDIA/AMD GPU detected. LocalAI will run in CPU-only mode.

运行

local-ai
9:49AM INF env file found, loading environment variables from file envFile=/etc/localai.env
9:49AM INF Setting logging to info
9:49AM INF Starting LocalAI using 16 threads, with models path: /usr/share/local-ai/models
9:49AM INF LocalAI version: v2.26.0 (09941c0bfb9119bb01a04b2a0a16897ecf2cd087)
9:49AM INF Preloading models from /usr/share/local-ai/models
9:49AM INF core/startup process completed!
9:49AM INF LocalAI API is listening! Please connect to the endpoint for API documentation. endpoint=http://0.0.0.0:8080

访问http://localhost:8080

安装model

访问http://localhost:8080/browse 浏览模型并选择安装
或者使用命令行，这里选择一个小模型

local-ai models install llama-3.2-1b-instruct:q4_k_m

https://hf-mirror.com/hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF

pom.xml

        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j-local-ai</artifactId>
            <version>1.0.0-beta1</version>
        </dependency>

controller

@RequestMapping("/local-ai")
@RestController
@Slf4j
public class LocalAiController {

    ChatLanguageModel model = LocalAiChatModel.builder()
            .baseUrl("http://localhost:8080/v1")
            .modelName("llama-3.2-1b-instruct:q4_k_m")
//            .maxTokens(3)
            .logRequests(true)
            .logResponses(true)
            .build();

    /**
     * curl -i http://localhost:8080/local-ai/chat?prompt=who are you
     * @param prompt
     * @return
     */
    @GetMapping("/chat")
    public String chat(@RequestParam("prompt") String prompt) {
        return model.chat(prompt);
    }

}

请求

http://localhost:8080/local-ai/chat?prompt=who%20are%20you
返回

I'm an AI assistant, which means I'm a computer program designed to understand and respond to human language. I'm here to help with a wide range of topics, answer questions, provide information, and even chat with you in a friendly and helpful way. I'm a large language model, which means I've been trained on a massive database of text from the internet, books, and other sources, which allows me to generate responses that are often surprisingly accurate and insightful. Some common ways I can assist you include: * Answering questions on a particular topic * Providing definitions and explanations * Offering suggestions and ideas * Summarizing long pieces of text * Translating languages (I can translate text from one language to another) * Offering writing and proofreading services * Generating text on a given topic or topic I'm always learning and improving, so please bear with me if I make any mistakes or don't quite understand the question. I'm here to help, so feel free to ask me anything that's on your mind!