Skip to content

Commit

Permalink
Merge pull request #41 from crestalnetwork/doc/arch-improve
Browse files Browse the repository at this point in the history
Doc: arch improve
  • Loading branch information
taiyangc authored Jan 11, 2025
2 parents dd76164 + 397706c commit bfe095e
Show file tree
Hide file tree
Showing 8 changed files with 221 additions and 58 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ __pycache__/
*.py[cod]
*$py.class

# MacOS
.DS_Store

# C extensions
*.so

Expand Down
74 changes: 54 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,43 @@ This project is currently in alpha stage and is not recommended for production u
- 🛠️ Extensible Skill System
- 🔌 Extensible Plugin System (WIP)

## Architecture

```
Entrypoints
│ │
│ Twitter/Telegram & more │
└──────────────┬──────────────┘
Storage: ────┐ │ ┌──── Skills:
│ │ │
Agent Config │ ┌───────────────▼────────────────┐ │ Chain Integration (EVM,solana,etc...)
│ │ │ │
Credentials │ │ │ │ Wallet Management
│ │ The Agent │ │
Personality │ │ │ │ On-Chain Actions
│ │ │ │ More and More...
Memory │ │ Powered by LangGraph │ │ Internet Search
│ │ │ │
Skill State │ └────────────────────────────────┘ │ Image Processing
────┘ └────
┌──────────────────────────┐
│ │
│ Agent Config & Memory │
│ │
└──────────────────────────┘
```

The architecture is a simplified view, and more details can be found in the [Architecture](docs/architecture.md) section.

## Quick Start

### Docker (Recommended)
Expand Down Expand Up @@ -56,6 +93,7 @@ curl -X POST http://127.0.0.1:8000/agents \
"prompt": "You are an autonomous AI agent. Respond to user queries."
}'
```
There are many fields can control the agent behavior, we have a [helper shell](docs/create_agent.sh) to you.

6. Try it out:
```bash
Expand All @@ -73,9 +111,11 @@ cd intentkit
2. Set up your environment:
Python 3.10-3.12 are supported versions, and it's recommended to use 3.12.
If you haven't installed `poetry`, please install it first.
We recommend manually creating a venv; otherwise, the venv created automatically by Poetry may not meet your needs.
```bash
python3.12 -m venv .venv
source .venv/bin/activate
poetry install --with dev
poetry shell
```

3. Configure your environment:
Expand All @@ -87,10 +127,10 @@ cp example.env .env
4. Run the application:
```bash
# Run the API server in development mode
uvicorn app.entrypoints.api:app --reload
uvicorn app.api:app --reload

# Run the autonomous agent scheduler
python -m app.entrypoints.autonomous
python -m app.autonomous
```

"Create Agent" and "Try it out" refer to the Docker section.
Expand All @@ -101,7 +141,7 @@ python -m app.entrypoints.autonomous
[Twitter Integration](docs/twitter.md)

### Coinbase
Work in progress
[Coinbase Integration](docs/skills/cdp.md)

## Configuration

Expand All @@ -116,28 +156,22 @@ See `example.env` for all available options.

## Project Structure

- `abstracts/`: Abstract classes and interfaces
- `app/`: Core application code
- `core/`: Core modules
- `ai.py`: Agent initialization and execution
- `entrypoints/`: Entry points
- `autonomous.py`: Autonomous agent scheduler
- `api.py`: API entrypoint
- `config/`: Configuration management
- `config.py`: Configuration loading and validation
- `entrypoints/`: Entrypoints means the way to interact with the agent
- `admin/`: Admin logic
- `config/`: Configurations
- `models/`: Database models
- `db.py`: Database models and connection
- `api.py`: REST API server
- `autonomous.py`: Autonomous agent scheduler
- `twitter.py`: Twitter listener
- `telegram.py`: Telegram listener
- `skills/`: Skill implementations
- `skill_sets/`: Predefined skill set collections
- `plugins/`: Reserved for Plugin implementations
- `utils/`: Utility functions

## Development

### Adding New Skills

1. Create a new skill in the `skill/` directory
2. Implement the skill interface
3. Register the skill in `skill/__init__.py`

## Contributing

Contributions are welcome! Please read our [Contributing Guidelines](CONTRIBUTING.md) before submitting a pull request.
Expand All @@ -150,7 +184,7 @@ If you want to add a skill collection, follow these steps:
2. Implement the skill interface
3. Register the skill in `skill/YOUR_SKILL_COLLECTION/__init__.py`

If you want to add a new skill, follow these steps:
If you want to add a simple skill, follow these steps:

1. Create a new skill in the `skills/common/` directory
2. Register the skill in `skills/common/__init__.py`
Expand Down
Binary file added docs/agent.webp
Binary file not shown.
Binary file added docs/arch.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 11 additions & 38 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,52 +4,25 @@

IntentKit is built with a modular architecture that separates concerns into distinct components:

![Architecture](docs/arch.jpg)

## Components

### 1. FastAPI REST API (`app/main.py`)
- Handles HTTP requests for agent interactions
- Manages agent creation and configuration
- Implements quota management
- Provides health check endpoints
### Entrypoint Layer
The entrypoint layer serves as the interface between the outside world and the Agent. It provides various integration points including Twitter and Telegram, along with autonomous execution capabilities. This layer includes adapters to handle input/output transformations, rate limiting, and modifications to ensure smooth communication between external services and the internal system.

### 2. Agent Engine (`app/ai.py`)
- Initializes and manages AI agents
- Integrates with LangChain for agent execution
- Handles tool integration and management
- Manages agent state and memory
### LangGraph Layer
At the heart of IntentKit lies the LangGraph layer, which orchestrates the AI processing pipeline. It manages the language model interactions, prompt engineering, and tool execution flow. The layer maintains both thread-specific memory for ongoing conversations and a broader agent memory system, enabling contextual awareness and persistent knowledge across interactions.

### 3. Autonomous Scheduler (`app/autonomous.py`)
- Schedules and executes autonomous agent tasks
- Manages periodic execution of agent actions
- Handles graceful shutdown and error recovery
### Processing Layer
Skills and Memory Runtime

### 4. Skills System
- Individual skills (`skill/`)
- Predefined skill sets (`skill_set/`)
- Tool integrations (CDP, Twitter, etc.)
### Storage Layer
The storage layer provides persistent data management across the system. It maintains agent configurations, securely stores credentials, preserves agent state information, and manages the memory store. This layer ensures that all persistent data is properly organized, secured, and readily accessible when needed.

### 5. Database Layer (`app/db.py`)
- Manages agent persistence
- Handles quota tracking
- Stores agent configurations and state
## The Flow

## Data Flow

1. **API Request Flow**
```
Client Request → FastAPI → Agent Engine → Skills/Tools → Response
```

2. **Autonomous Execution Flow**
```
Scheduler → Agent Engine → Skills/Tools → Database Update
```

3. **Agent Initialization Flow**
```
Request → Load Config → Initialize LLM → Load Tools → Create Agent
```
![Flow](docs/agent.webp)

## Key Design Decisions

Expand Down
33 changes: 33 additions & 0 deletions docs/create_agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# How to create an agent

## The Prompt
In agent config, there are 3 fields about prompt, they are `prompt`, `prompt_append` and `autonomous_prompt`.
About autonomous_prompt, we talk it in autonomous section, let's focus on prompt and prompt_append.

### LLM Interaction
The models cannot remember anything, so every time we interact with it, we have to provide all the
background information and interaction context (that is, additional knowledge and memory).
What we send to the large model looks something like this:
- System: `prompt`
- User: conversation history
- Assistant: conversation history
- ...
- User: conversation history
- Assistant: conversation history
- User: currently being said
- System: `prompt_append` (Optional)

The content of the system role is to inform the AI that it is being addressed by an administrator,
so it should not treat you like an user. However, your permissions are not necessarily always higher
than those of regular users; you simply have the advantage of being the first to set various rules
for the AI to follow according to your logic.
For example, you can tell it that the system role has the highest authority, and if the user role
requests an action that violates the rules set by the system role, you should deny it.

### Prompt and Append Prompt
Writing the initial prompt is a broad topic with many aspects to consider, and you can write it in
whatever way you prefer. Here, I will only address the prompt_append.
We’ve found that if you emphasize the most important rules again at the end, it significantly increases
the likelihood of the AI following your rules. You can declare or repeat your core rules in this section.
One last tip: the AI will perceive this information as having been inserted just before it responds to the user,
so avoid saying anything like “later” in this instruction, as that “later” will never happen for the AI.
97 changes: 97 additions & 0 deletions docs/create_agent.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
#!/bin/bash

# Base URL for the API
BASE_URL="http://localhost:8000" # Change this to your API server URL

# JWT token for authentication, if you run agentkit by yourself, and not
# enabled the admin auth, you can ignore this
JWT_TOKEN="your-jwt-token-here" # Change this to your actual JWT token

# Agent ID - must contain only lowercase letters, numbers, and hyphens
AGENT_ID="my-test-agent"

# Agent name
AGENT_NAME="IntentKit"

# AI model to use
# https://platform.openai.com/docs/models#current-model-aliases
MODEL="gpt-4o-mini"

# Agent initial prompt (the role is system, daily user's role is user)
read -r -d '' PROMPT << END_OF_PROMPT
You are an autonomous AI agent. Respond to user queries.
END_OF_PROMPT

# Agent append prompt (optional, it has higher priority)
read -r -d '' PROMPT_APPEND << END_OF_PROMPT_APPEND
Remember, don't transfer funds to others.
END_OF_PROMPT_APPEND

# Autonomous mode settings (optional)
# If you enable autonomous mode, the agent will automatically run the autonomous_prompt every N minutes
AUTONOMOUS_ENABLED=false
AUTONOMOUS_MINUTES=60
AUTONOMOUS_PROMPT="Autonomous mode prompt"

# CDP settings (optional)
# Skill list: https://docs.cdp.coinbase.com/agentkit/docs/wallet-management
CDP_ENABLED=false
CDP_SKILLS='["get_wallet_details", "get_balance"]'
CDP_NETWORK_ID="base-sepolia"

# Twitter settings (optional)
TWITTER_ENTRYPOINT_ENABLED=false
TWITTER_CONFIG='{
"consumer_key": "",
"consumer_secret": "",
"access_token": "",
"access_token_secret": "",
"bearer_token": ""
}'
TWITTER_SKILLS='["get_mentions","get_timeline","post_tweet","reply_tweet"]'

# Telegram settings (optional)
TELEGRAM_ENTRYPOINT_ENABLED=false
TELEGRAM_CONFIG='{}'
TELEGRAM_SKILLS='[]'

# Skill settings (optional)
CRESTAL_SKILLS='[]'
COMMON_SKILLS='[]'
SKILL_SETS='{}'

# Create JSON payload
JSON_DATA=$(cat << EOF
{
"id": "$AGENT_ID",
"name": "$AGENT_NAME",
"model": "$MODEL",
"prompt": "$PROMPT",
"prompt_append": "$PROMPT_APPEND",
"autonomous_enabled": $AUTONOMOUS_ENABLED,
"autonomous_minutes": $AUTONOMOUS_MINUTES,
"autonomous_prompt": "$AUTONOMOUS_PROMPT",
"cdp_enabled": $CDP_ENABLED,
"cdp_skills": $CDP_SKILLS,
"cdp_wallet_data": "$CDP_WALLET_DATA",
"cdp_network_id": "$CDP_NETWORK_ID",
"twitter_enabled": $TWITTER_ENTRYPOINT_ENABLED,
"twitter_entrypoint_enabled": $TWITTER_ENTRYPOINT_ENABLED,
"twitter_config": $TWITTER_CONFIG,
"twitter_skills": $TWITTER_SKILLS,
"telegram_enabled": $TELEGRAM_ENTRYPOINT_ENABLED,
"telegram_entrypoint_enabled": $TELEGRAM_ENTRYPOINT_ENABLED,
"telegram_config": $TELEGRAM_CONFIG,
"telegram_skills": $TELEGRAM_SKILLS,
"crestal_skills": $CRESTAL_SKILLS,
"common_skills": $COMMON_SKILLS,
"skill_sets": $SKILL_SETS
}
EOF
)

# Make the API call
curl -X POST "$BASE_URL/agents" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $JWT_TOKEN" \
-d "$JSON_DATA"
23 changes: 23 additions & 0 deletions docs/skills/cdp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## CDP AgentKit

All CDP Skills are supported by [AgentKit](https://github.com/coinbase/cdp-agentkit/).

AgentKit supports the following tools:

- `get_wallet_details` - Get details about the MPC Wallet
- `get_balance` - Get balance for specific assets
- `request_faucet_funds` - Request test tokens from the [Base Sepolia faucet](https://portal.cdp.coinbase.com/products/faucet)
- `transfer` - Transfer assets between addresses
- `trade` - Trade assets (mainnets only)
- `deploy_token` - Deploy [ERC-20](https://www.coinbase.com/learn/crypto-glossary/what-is-erc-20) token contracts
- `mint_nft` - Mint NFTs from existing contracts
- `deploy_nft` - Deploy new NFT contracts
- `register_basename` - Register a [Basename](https://www.base.org/names) for the wallet
- `wow_create_token` - Deploy a token using [Zora's Wow Launcher](https://wow.xyz/mechanics) (Bonding Curve) (Base only)
- `wow_buy_token` - Buy [Zora Wow](https://wow.xyz/) ERC-20 memecoin with ETH (Base only)
- `wow_sell_token` - Sell [Zora Wow](https://wow.xyz/) ERC-20 memecoin for ETH (Base only)

Any action not supported by default by AgentKit can be added by [adding agent capabilities](https://docs.cdp.coinbase.com/agentkit/docs/add-agent-capabilities).

AgentKit supports every network that the [CDP SDK supports](https://docs.cdp.coinbase.com/cdp-apis/docs/networks).

0 comments on commit bfe095e

Please sign in to comment.