Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 2.0 #21

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,3 @@
members = [
"ata"
]

[profile.release.package.ata]
strip = true
Comment on lines -5 to -7
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed? This it cause errors on Android?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure. It may be a bug in cargo add, it's also possible that I messed something up when editing these files in vim on Android 😂😂

13 changes: 11 additions & 2 deletions ata/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
[package]
name = "ata"
version = "1.0.3"
version = "2.0.0"
edition = "2021"
authors = ["Rik Huijzer <[email protected]>", "Fredrick R. Brennan <[email protected]>", "ATA Project Authors"]
license = "MIT"

[dependencies]
rustyline = "10.1"
Expand All @@ -11,7 +13,14 @@ serde = { version = "1", features = ["derive"] }
serde_json = { version = "1" }
tokio = { version = "1", features = ["full"] }
toml = { version = "0.6" }
clap = { version = "4.1.4", features = ["derive"] }
clap = { version = "4.1.4", features = ["derive", "cargo"] }
log = "0.4"
env_logger = "0.10"
directories = "4.0.1"
lazy_static = "1.4.0"
os_str_bytes = "6.4.1"
bevy_reflect = "0.9.1"
bevy_utils = "0.9.1"

[dev-dependencies]
pretty_assertions = "1"
28 changes: 28 additions & 0 deletions ata/src/args.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
use crate::config::ConfigLocation;

use clap::Parser;

/// Ask the Terminal Anything (ATA): OpenAI GPT in the terminal
#[derive(Parser, Debug)]
#[command(author = crate_authors!("\n\t"), version = crate_version!(),
about, long_about = None,
help_template = "{before-help}{name} {version} — {about}\
\n\n\
© 2023\t{author}\
\n\n\
{usage-heading} {usage}\
\n\n\
{all-args}{after-help}")]
pub struct Ata {
/// Path to the configuration TOML file.
#[arg(short = 'c', long = "config", default_value = "")]
pub config: ConfigLocation,

/// Avoid printing the configuration to stdout.
#[arg(long)]
pub hide_config: bool,

/// Print the keyboard shortcuts.
#[arg(long)]
pub print_shortcuts: bool,
}
250 changes: 250 additions & 0 deletions ata/src/config.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
use std::convert::Infallible;
use std::ffi::OsString;
use std::fmt::{self, Display};

use std::path::{Path, PathBuf};
use std::str::FromStr;

use bevy_reflect::{Reflect, Struct as _};
use bevy_utils::HashMap;
use directories::ProjectDirs;
use os_str_bytes::OsStrBytes as _;
use os_str_bytes::OsStringBytes as _;
use serde::{Deserialize, Serialize};
use toml::de::Error as TomlError;

lazy_static! {
static ref DEFAULT_CONFIG_FILENAME: PathBuf = "ata.toml".into();
}

// For definitions, see https://platform.openai.com/docs/api-reference/completions/create
#[repr(C)]
#[derive(Clone, Deserialize, Debug, Serialize, Reflect)]
#[serde(default)]
pub struct Config {
pub api_key: String,
pub model: String,
pub max_tokens: i64,
pub temperature: f64,
pub suffix: Option<String>,
pub top_p: f64,
pub n: u64,
pub stream: bool,
pub logprobs: u8,
pub echo: bool,
pub stop: Vec<String>,
pub presence_penalty: f64,
pub frequency_penalty: f64,
pub best_of: u64,
pub logit_bias: HashMap<String, f64>,
}

impl Config {
pub fn validate(&self) -> Result<(), String> {
if self.api_key.is_empty() {
return Err(String::from("API key is missing"));
}

if self.model.is_empty() {
return Err(String::from("Model ID is missing"));
}

if self.max_tokens < 1 || self.max_tokens > 2048 {
return Err(String::from("Max tokens must be between 1 and 2048"));
}

if self.temperature < 0.0 || self.temperature > 1.0 {
return Err(String::from("Temperature must be between 0.0 and 1.0"));
}

if let Some(suffix) = &self.suffix {
if suffix.is_empty() {
return Err(String::from("Suffix cannot be an empty string"));
}
}

if self.top_p < 0.0 || self.top_p > 1.0 {
return Err(String::from("Top-p must be between 0.0 and 1.0"));
}

if self.n < 1 || self.n > 10 {
return Err(String::from("n must be between 1 and 10"));
}

if self.logprobs > 2 {
return Err(String::from("logprobs must be 0, 1, or 2"));
}

if self.stop.iter().any(|stop| stop.is_empty()) || self.stop.len() > 4 {
return Err(String::from("Stop phrases cannot contain empties"));
}

if self.presence_penalty < 0.0 || self.presence_penalty > 1.0 {
return Err(String::from("Presence penalty must be between 0.0 and 1.0"));
}

if self.frequency_penalty < 0.0 || self.frequency_penalty > 1.0 {
return Err(String::from(
"Frequency penalty must be between 0.0 and 1.0",
));
}

if self.best_of < 1 || self.best_of > 5 {
return Err(String::from("best_of must be between 1 and 5"));
}

for (key, value) in &self.logit_bias {
if value < &-2.0 || value > &2.0 {
return Err(format!(
"logit_bias for {} must be between -2.0 and 2.0",
key
));
}
}

Ok(())
}
}

impl Default for Config {
fn default() -> Self {
Self {
model: "text-davinci-003".into(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things move quickly in OpenAI API's and since the release of the ChatGPT models, I think those should be the primary. I think that v2 of ata should just drop support for non-chat models for simplicity. What do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wonder whether people actually would use the settings such as top_p. For people who want to use that, there is https://platform.openai.com/playground. ata aims more at productivity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well that's why I added the ability to also specify configuration files by name. That way you can load an alternate one for a certain project.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I wanted to make it so that every single configuration argument could just as well be specified via the command line or the operating system environment and then in that case you don't even need to use configuration files. So the order of loading would be, configuration files if any, environment, arguments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that v2 of ata should just drop support for non-chat models for simplicity. What do you think?

Sorry that's really basically the opposite of what I had in mind. So the reason that this project was done on Android (part of the idea was that I would only actually work on things that would make using a rooted Android phone better) is that the playground website is extremely broken in the Android browsers. The streaming feature barely works, it always loses your text and doesn't get it back through undo etc.

I also wanted to be able to pipe to ata. I did not quite finish that yet but I will.

As I see it, ata is most useful as a generic console application which supports all of the GPT APIs. And much like most other console applications, that means that it has an interactive mode and a batch mode.

I think that if you want to use ChatGPT, even by default, that's no problem. But I would not drop support for the existing API, I would just put it behind an option.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for giving more details. I have been thinking about it for a bit more and I see your point. For me, the main goal of ata was also to provide a more productive interface to ChatGPT than is currently available at chat.openai.com.

As I see it, ata is most useful as a generic console application which supports all of the GPT APIs. And much like most other console applications, that means that it has an interactive mode and a batch mode.

I think that if you want to use ChatGPT, even by default, that's no problem. But I would not drop support for the existing API, I would just put it behind an option.

Yes I agree that that would be best. The problem for me is that I do not have time to support all GPT APIs. In the ideal case, I would start adding tests for example. I currently do not have time for that since my PhD project also has a lot of deadlines currently. At the same time, I do want to try to keep up with OpenAI while they are moving at lightning speed. So, the most useful middle ground there in my opinion is then to drop support for older models and only support the, much cheaper, chatgpt models. That way, Android and terminal users can have productive interactions with at least one GPT model

Copy link
Collaborator

@rikhuijzer rikhuijzer Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the ideal case, I would start adding tests for example.

More specifically, the complexity of the project is nearing my own capabilities. Without tests, I cannot increase complexity much more or I would start introducing more and more bugs. However, tests are complex to make too since it would require a mock stdout (for example, like kkawakam/rustyline#652 (comment)) and a mock OpenAI server. Both are possible, but take time which I don't have at this moment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine, I think we should just fork. I'm not really going to add test because I don't really care if it works or not for every user in every use case... They can report issues and then I'll fix those. My plan certainly is not to have a mock server and a mock standard out.

max_tokens: 16,
temperature: 0.5,
suffix: None,
top_p: 1.0,
n: 1,
stream: false,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream has to be enabled to enable streaming. That's one of the main features of ata.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I don't want always want streaming and want to pipe to it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I don't want always want streaming and want to pipe to it.

How about a Bash script wrapper around curl? Something like

#!/usr/bin/env bash

RESPONSE=$(curl --silent https://api.openai.com/v1/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer <API KEY>" \
    -d "{\"model\": \"text-davinci-003\", \"prompt\": \"$@\", \"temperature\": 0, \"max_tokens\": 7}")

echo $RESPONSE | jq '.choices[0].text'

For example, this gives:

$ bash tmp.sh 'hi there'
"\n\nHello! How are you"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that's possible but wanted an application to do it.

logprobs: 0,
echo: false,
stop: vec![],
presence_penalty: 0.0,
frequency_penalty: 0.0,
best_of: 1,
logit_bias: HashMap::new(),
api_key: String::default(),
}
}
}

#[derive(Clone, Deserialize, Debug, Default)]
pub enum ConfigLocation {
#[default]
Auto,
Path(PathBuf),
Named(PathBuf),
}

impl FromStr for ConfigLocation {
type Err = Infallible;

fn from_str(s: &str) -> Result<Self, Self::Err> {
Ok(if !s.contains(".") && s.len() > 0 {
Self::Named(s.into())
} else if s.trim().len() > 0 {
Self::Path(s.into())
} else if s.trim().is_empty() {
Self::Auto
} else {
unreachable!()
})
}
}

impl<S> From<S> for ConfigLocation
where
S: AsRef<str>,
{
fn from(s: S) -> Self {
Self::from_str(s.as_ref()).unwrap()
}
}

fn get_config_dir() -> PathBuf {
ProjectDirs::from(
"ata",
"Ask the Terminal Anything (ATA) Project Authors",
"ata",
)
.unwrap()
.config_dir()
.into()
}

pub fn default_path(name: Option<&Path>) -> PathBuf {
let mut config_file = get_config_dir().to_path_buf();
let file: Vec<_> = if let Some(name) = name {
let mut name = name.to_path_buf();
name.set_extension("toml");
name.as_os_str()
.to_raw_bytes()
.into_iter()
.map(|i| *i)
.collect()
} else {
let name = DEFAULT_CONFIG_FILENAME.to_string_lossy();
name.bytes().collect()
};
let file = OsString::assert_from_raw_vec(file);
config_file.push(&file);
config_file
}

impl ConfigLocation {
pub fn location(&self) -> PathBuf {
match self {
ConfigLocation::Auto => {
let config_dir = get_config_dir().to_path_buf();
if DEFAULT_CONFIG_FILENAME.exists() {
warn!(
"{} found in working directory BUT UNSPECIFIED. \
This behavior is DEPRECATED. \
Please move it to {}.",
DEFAULT_CONFIG_FILENAME.display(),
config_dir.display()
);
return DEFAULT_CONFIG_FILENAME.clone();
}
default_path(None)
}
ConfigLocation::Path(pb) => pb.clone(),
ConfigLocation::Named(name) => default_path(Some(name)),
}
}
}

impl FromStr for Config {
type Err = TomlError;

fn from_str(contents: &str) -> Result<Self, Self::Err> {
toml::from_str(&contents)
}
}

impl<S> From<S> for Config
where
S: AsRef<str>,
{
fn from(s: S) -> Self {
Self::from_str(s.as_ref()).unwrap_or_else(|e| panic!("Config parsing failure!: {:?}", e))
}
}

impl Display for Config {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> Result<(), fmt::Error> {
let mut ok = Ok(());
for (i, value) in self.iter_fields().enumerate() {
if !ok.is_ok() {
break;
}
let value: &dyn Reflect = value;
let key = self.name_at(i).unwrap();
if key == "api_key" {
continue
}
ok = writeln!(f, "{key}: {:#?}", value);
}
ok
}
}
51 changes: 42 additions & 9 deletions ata/src/help.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
use rustyline::Editor;

use crate::config;
use std::fs::{self, File};
use std::io::Write as _;

pub fn commands() {
println!("
Ctrl-A, Home Move cursor to the beginning of line
Expand Down Expand Up @@ -29,18 +35,21 @@ Thanks to <https://github.com/kkawakam/rustyline#emacs-mode-default-mode>.
");
}

pub fn missing_toml(args: Vec<String>) {
const EXAMPLE_TOML: &str = r#"api_key = # "<YOUR SECRET API KEY>"
model = "text-davinci-003"
max_tokens = 500
temperature = 0.8"#;

pub fn missing_toml() {
let default_path = config::default_path(None);
eprintln!(
r#"
Could not find the file `ata.toml`. To fix this, use `{} --config=<Path to ata.toml>` or have `ata.toml` in the current dir.
Could not find the file `ata.toml`. To fix this, create {0}.

For example, make a new file `ata.toml` in the current directory with the following content (the text between the ```):
For example, use the following content (the text between the ```):

```
api_key = "<YOUR SECRET API KEY>"
model = "text-davinci-003"
max_tokens = 500
temperature = 0.8
{EXAMPLE_TOML}
```

Here, replace `<YOUR SECRET API KEY>` with your API key, which you can request via https://beta.openai.com/account/api-keys.
Expand All @@ -49,9 +58,33 @@ The `max_tokens` sets the maximum amount of tokens that the server will answer w

The `temperature` sets the `sampling temperature`. From the OpenAI API docs: "What sampling temperature to use. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer." According to Stephen Wolfram [1], setting it to a higher value such as 0.8 will likely work best in practice.


[1]: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

"#, args[0]);
"#,
(&default_path).display()
);
let mut rl = Editor::<()>::new().unwrap();
eprintln!(
"Do you want me to write this example file to {0} for you to edit?",
(&default_path).display()
);
let readline = rl.readline("[y/N] ");
if let Ok(msg) = readline {
if msg
.trim()
.chars()
.nth(0)
.map(|c| c.to_lowercase().collect::<String>() == "y")
.unwrap_or(false)
{
if !default_path.exists() && !default_path.parent().unwrap().is_dir() {
fs::create_dir_all(&default_path).expect("Could not make configuration directory");
}
let mut f = File::create(&default_path).expect("Unable to create file");
f.write_all(EXAMPLE_TOML.as_bytes())
.expect("Unable to write to file");
}
}
std::process::exit(1);
}

Loading