Skillget

Octet.Chat

agent
Guvenlik Denetimi
Gecti
Health Gecti
  • License — License: MIT
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Community trust — 13 GitHub stars
Code Gecti
  • Code scan — Scanned 8 files during light audit, no dangerous patterns found
Permissions Gecti
  • Permissions — No dangerous permissions requested
Purpose
This project is a Java-based framework designed to deploy private AI agents and LLM services. It allows users to run local models (such as Llama 3) with features like continuous conversation, parallel inference, and function calling.

Security Assessment
Overall Risk: Medium. The repository does not request dangerous permissions, contains no hardcoded secrets, and a light code scan found no malicious patterns. However, because it is designed to launch local API servers and specifically implements "Function calling," there is an inherent risk. Function calling allows an LLM to execute commands or interact with external systems, which could lead to unpredictable behavior if the AI character or prompt is not strictly managed. Additionally, running the web server exposes endpoints that should be properly firewalled to prevent unauthorized access.

Quality Assessment
The project appears active and well-maintained, with repository pushes as recent as today. It is open-source under the permissive and standard MIT license. Community trust is currently small but present, indicated by 13 GitHub stars. The documentation is thorough, providing clear setup instructions and API usage examples for both desktop and server environments.

Verdict
Use with caution due to the inherent risks of AI function calling, though the code itself is transparent and seems safe to deploy in a controlled environment.
SUMMARY

Build your own private auto agent using Octet.Chat.

README.md

🚀 Octet.Chat

CI
Maven Central
README Zh_CN
GitHub

This is a LLMs project implemented in Java.
You can use it to deploy your own private services, supports the Llama3 and GPT models and other open-source models.

Provides

  • llama-java-core Simple Java library
  • octet-chat-app Octet desktop
  • octet-chat-app API servers

Features

  • 🦙 Built on llama.cpp
  • 😊 Support AI Agent and implements Function calling based on Qwen-chat
  • 🤖 Supports parallel inference, continuous conversation and text generation
  • 📦 Supports the Llama3 and GPT models, such as Baichuan 7B,Qwen 7B
Last updated

...

  • 🚀 Supported dynamic temperature sampling.
  • 🚀 Added WebUI to octet-chat-app.
  • 🚀 Updated API parameters.
  • 🚀 Optimized chat formatter and Windows Cli.
  • 🚀 Refactored function calls, Optimized chat formatter and APIs.

Quick start

[!NOTE]

You can quantify the original model yourself or search for huggingface to obtain open-source models.

🖥 Octet desktop

Octet Agent

💡 API servers

How to use

Set a custom AI character and Launch the app, open browser enjoy it now http://YOUR_IP_ADDR:8152/

# Default URL: http://YOUR_IP_ADDR:8152/

cd <YOUR_PATH>/octet-chat-app
bash app_server.sh start YOUR_CHARACTER

[!TIP]

It can be integrated into your services, such as VsCode, App, Wechat, etc.

How to call API

Api docs: http://127.0.0.1:8152/swagger-ui.html

curl --location 'http://127.0.0.1:8152/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "messages": [
        {
            "role": "USER",
            "content": "Who are you?"
        }
    ],
    "user": "User",
    "stream": true
}'

The API will return data in a stream format:

{
    "id": "octetchat-98fhd2dvj7",
    "model": "Llama2-chat",
    "created": 1695614393810,
    "choices": [
        {
            "index": 0,
            "delta": {
                "content": "Hi"
            },
            "finish_reason": "NONE"
        }
    ]
}

Documentation

Development

Characters config

Disclaimer

[!IMPORTANT]

  • This project does not provide any models. Please obtain the model files yourself and comply with relevant agreements.
  • Please do not use this project for illegal purposes, including but not limited to commercial use, profit-making use, or use that violates laws and regulations.
  • Any legal liability arising from the use of this project shall be borne by the user, and this project shall not bear any legal liability.

Feedback

  • If you have any questions, please submit them in GitHub Issue.

Yorumlar (0)

Sonuc bulunamadi