Skip to content

Commit

Permalink
conf: move to relative path.
Browse files Browse the repository at this point in the history
  • Loading branch information
kfysck committed Sep 7, 2022
1 parent 1e80586 commit 8f7c63d
Show file tree
Hide file tree
Showing 31 changed files with 52 additions and 60 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ jobs:
rm dtc/sharding.tar.gz
cp ../dockerfiles/devel/mysql-connector-java-5.1.49.jar dtc/sharding/lib/mysql-connector-java-5.1.49.jar
cp ../conf/log4cplus.conf dtc/log/
tar -zcvf dtc.tar.gz dtc/
- name: Create Release
Expand Down
2 changes: 1 addition & 1 deletion conf/log4cplus.conf
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ log4cplus.appender.STDOUT.layout.ConversionPattern=[%T] %D{%Y-%m-%d %H:%M:%S} %-

## synchronous log properties.
log4cplus.appender.apdPlatform=log4cplus::TimeBasedRollingFileAppender
log4cplus.appender.apdPlatform.FilenamePattern=/var/log/dtc/dtc.%d{yyyyMMdd}.log
log4cplus.appender.apdPlatform.FilenamePattern=../log/dtc/dtc.%d{yyyyMMdd}.log
log4cplus.appender.apdPlatform.Append=true
log4cplus.appender.apdPlatform.MaxHistory=999
log4cplus.appender.apdPlatform.ImmediateFlush=false
Expand Down
4 changes: 2 additions & 2 deletions dockerfiles/agent/dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
FROM centos:centos8

ARG basepath=/usr/local/dtc
ARG confpath=/etc/dtc
ARG logpath=/var/log/dtc
ARG confpath=$basepath/conf
ARG logpath=$basepath/log

RUN mkdir -p $basepath/bin
RUN mkdir -p $confpath
Expand Down
8 changes: 2 additions & 6 deletions dockerfiles/devel/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ sleep_count=0
while [ $sleep_count -le 100 ]
do
if [ -f "/usr/local/dtc/conf/dtc.yaml" ]; then
mkdir /etc/dtc
cp /usr/local/dtc/conf/log4cplus.conf /etc/dtc
mkdir /usr/local/dtc/conf
cp /usr/local/dtc/conf/log4cplus.conf /usr/local/dtc/conf/
echo "Start running process: "$DTC_BIN","$DTC_ARGV
cd /usr/local/dtc/bin/
./$DTC_BIN $DTC_ARGV
Expand All @@ -18,7 +18,3 @@ do
fi
done
echo "Timeout waitting for dtc conf files."

#cd /usr/local/lib/
#ln -snf libdtc.so.1 libdtc.so
#ln -snf libdtcapi.so libdtc.so.1
4 changes: 1 addition & 3 deletions dockerfiles/server/dockerfile
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
FROM centos:centos8

ARG basepath=/usr/local/dtc
ARG confpath=/etc/dtc
ARG logpath=/var/log/dtc
ARG confpath=$basepath/conf

RUN mkdir -p $basepath/bin
RUN mkdir -p $basepath/data
RUN mkdir -p $basepath/stat
RUN mkdir -p $basepath/log
RUN mkdir -p $confpath
RUN mkdir -p $logpath

COPY core $basepath/bin/core
COPY hwcserver $basepath/bin/hwcserver
Expand Down
15 changes: 7 additions & 8 deletions docs/configure.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@

## 配置文件
配置文件目录:/etc/dtc<br/>
配置文件目录:conf/<br/>
- AGENT服务的配置文件是:
* agent.xml 指定后端dtc server的主、备服务的地址信息和权重;
- DTC服务的配置文件是:
* dtc.yaml 配置缓存模式,缓存大小等。
* table.yaml 表结构的定义、数据库连接信息在此配置。
* dtc.yaml dtc模块的配置文件,包括基础配置和表结构信息。

下面就各配置文件具体配置选项做介绍:
### agent.xml
Expand Down Expand Up @@ -38,10 +37,7 @@ agent配置文件位置为当前项目的conf/agent.xml,主要配置以下字
* sharding:
该字段用户设置分库分表的信息,key字段用于设置依照此字段进行分片。table字段用于设置分表的信息,例如分表名为opensource_0/opensource_1.....opensouce_9,则只需要设置为{prefix: [*table, _], start: 0, last: 9}

#### extension 扩展库,提供多租户功能
在dtc的基础缓存和数据代理功能之外,还提供了扩展库,通过配置此模块能够在数据库中进行复杂查询和分库分表功能。

### table.yaml
#### table
cache配置文件位置为当前项目的conf/table.yaml,主要配置以下字段:
* TABLE_CONF.table_name
* TABLE_CONF.field_count
Expand All @@ -50,4 +46,7 @@ cache配置文件位置为当前项目的conf/table.yaml,主要配置以下字
* FIELD*ID*.field_name 字段名
* FIELD*ID*.field_type 字段类型:1.整数型 2.无符号整数形 3.浮点型 4.字符串(大小写不明感) 5.字符串(大小写敏感)
* FIELD*ID*.field_size 字段长度。定义了该字段的大小。例如int型可配置为4字节,long型可配置为8字节,字符串类型可根据具体使用场景配置长度,但最大长度不得超过64KB。
* FIELD*ID*.field_unique [可选]默认值0,可配置0或1。当为0时表示该字段的值不唯一,1时表示该字段值唯一。
* FIELD*ID*.field_unique [可选]默认值0,可配置0或1。当为0时表示该字段的值不唯一,1时表示该字段值唯一。

#### extension 扩展库,提供多租户功能
在dtc的基础缓存和数据代理功能之外,还提供了扩展库,通过配置此模块能够在数据库中进行复杂查询和分库分表功能。
11 changes: 5 additions & 6 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
## 目录结构
Bin文件目录:/usr/local/dtc<br/>
配置文件目录:/etc/dtc<br/>
日志目录:/var/log/<br/>
配置文件目录:/usr/local/dtc/conf/<br/>
日志目录:/usr/local/dtc/log/<br/>
统计数据目录:/usr/local/dtc/stat/<br/>
binlog数据目录:/usr/local/log/<br/>
binlog数据目录:/usr/local/dtc/log/<br/>

## DTC模式

Expand All @@ -29,7 +29,7 @@ demo中定义的表名为dtc_opensource, <br/>
为了省去配置环境的麻烦,Demo中提供docker镜像,直接运行即可启动服务端:<br/>
```shell
docker pull dtc8/dtc:latest
docker run --rm --name dtc -p <MY_LISTENER_PORT>:12001 -v <MY_HOST_CONF_DIR>:/etc/dtc/ -e DTC_BIN=dtc -e DTC_ARGV=-ayc dtc8/dtc
docker run --rm --name dtc -p <MY_LISTENER_PORT>:12001 -v <MY_HOST_CONF_DIR>:/usr/local/dtc/conf/ -e DTC_BIN=dtc -e DTC_ARGV=-ayc dtc8/dtc
```

## 运行Client测试示例
Expand Down Expand Up @@ -78,8 +78,7 @@ mkdir -p basepath
mkdir -p /usr/local/dtc/data
mkdir -p /usr/local/dtc/stat
mkdir -p /usr/local/dtc/log
mkdir -p /etc/dtc
mkdir -p /var/log/dtc
mkdir -p /usr/local/dtc/conf
```
* 将bin文件拷贝到/usr/local/dtc文件夹,并赋执行权限
```
Expand Down
2 changes: 1 addition & 1 deletion src/agent-watchdog/async_conn_entry.cc
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
#include <unistd.h>
#include "log.h"

#define ROOT_PATH "/etc/dtc/"
#define ROOT_PATH "../conf/"
const char *fulldata_name = "async-connector";
extern std::map<std::string, std::string> map_dtc_conf; //key:value --> dtc addr:conf file name

Expand Down
4 changes: 2 additions & 2 deletions src/agent-watchdog/daemons.cc
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,9 @@ void close_sharding()
{
char* p[1];
p[0] = NULL;
execv("/usr/local/agent/sharding/bin/stop.sh", p);
execv("../sharding/bin/stop.sh", p);

system("/usr/local/agent/sharding/bin/stop.sh");
system("../sharding/bin/stop.sh");
}

void WatchDog::run_loop()
Expand Down
2 changes: 1 addition & 1 deletion src/agent-watchdog/log.h
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ using namespace log4cplus::helpers;

static Logger logger = Logger::getRoot();

#define LOG4CPLUS_CONF_FILE "/etc/dtc/log4cplus.conf"
#define LOG4CPLUS_CONF_FILE "../conf/log4cplus.conf"

/*********************************************
**TRACE:调试应用的详细步骤
Expand Down
4 changes: 2 additions & 2 deletions src/agent-watchdog/main.cc
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@

extern char cache_file[256];
extern char table_file[256];
#define ROOT_PATH "/etc/dtc/"
char agent_file[256] = "/etc/dtc/agent.xml";
#define ROOT_PATH "../conf/"
char agent_file[256] = "../conf/agent.xml";
std::map<std::string, std::string> map_dtc_conf; //key:value --> dtc addr:conf file name

#define DA_VERSION_MAJOR 1
Expand Down
6 changes: 3 additions & 3 deletions src/agent-watchdog/sharding_entry.cc
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,13 @@ void ShardingEntry::exec()
{
char *argv[4];

int ret = system("/usr/local/agent/bin/conf-gen-utils");
int ret = system("./conf-gen-utils");
if(ret == 0)
{
set_proc_title("agent_sharding");
argv[0] = (char *)"/usr/local/agent/sharding/bin/start.sh";
argv[0] = (char *)"../sharding/bin/start.sh";
argv[1] = "3307";
argv[2] = "/etc/dtc";
argv[2] = "../conf";
argv[3] = NULL;
execv(argv[0], argv);
}
Expand Down
6 changes: 3 additions & 3 deletions src/agent/da.c
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@
#define DA_VERSION_DETAIL \
DA_VERSION_STR" built at: "__DATE__" "__TIME__

#define DA_CONF_PATH "/etc/dtc/agent.xml"
#define DA_LOG_DIR "/var/log/dtc"
#define DA_CONF_PATH "../conf/agent.xml"
#define DA_LOG_DIR "../log"

#define DA_LOG_DEFAULT 3
#define DA_MBUF_SIZE DEF_MBUF_SIZE
Expand Down Expand Up @@ -344,7 +344,7 @@ static void show_usage(void) {
write_stderr(
" -v, --verbosity=N : set logging level (default: 3, min: 1, max: 7)" CRLF
" -o, --output=S : set logging dir (default: /var/log/dtc/)" CRLF
" -c, --conf-file=S : set configuration file (default: /etc/dtc/agent.xml)" CRLF
" -c, --conf-file=S : set configuration file (default: ../conf/agent.xml)" CRLF
" -e, --event-max-timeout=S : set epoll max timeout(ms)(default: (30*1000)ms)" CRLF
" -i, --stats_interval=S : set stats aggregator interval(ms)(default: (10*1000)ms)" CRLF
" -p, --pid-file=S : set pid file (default: off)" CRLF
Expand Down
2 changes: 1 addition & 1 deletion src/agent/my/my_parse.c
Original file line number Diff line number Diff line change
Expand Up @@ -660,7 +660,7 @@ int my_get_route_key(uint8_t *sql, int sql_len, int *start_offset,
char conf_path[260] = {0};
if(mid != 0)
{
sprintf(conf_path, "/etc/dtc/dtc-conf-%d.yaml", mid);
sprintf(conf_path, "../conf/dtc-conf-%d.yaml", mid);
r->mid = mid;
}

Expand Down
2 changes: 1 addition & 1 deletion src/complex/main.cc
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ static PollerBase* workerThread;
CTransactionGroup* FullDBGroup = NULL;
CTransactionGroup* HotDBGroup = NULL;

std::string conf_path = "/etc/dtc/dtc.yaml";
std::string conf_path = "../dtc.yaml";

static int start_main_thread()
{
Expand Down
2 changes: 1 addition & 1 deletion src/daemons/helper.cc
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ void WatchDogHelper::exec()
if (conf_ == DBHELPER_TABLE_NEW) {
argv[argc++] = (char *)"-t";// 4
char tableName[64];
snprintf(tableName, 64, "/etc/dtc/dtc%d.yaml", num_);
snprintf(tableName, 64, "../conf/dtc%d.yaml", num_);
argv[argc++] = tableName; // 5
} else if (conf_ == DBHELPER_TABLE_ORIGIN &&
strcmp(daemons_table_file, TABLE_CONF_NAME)) {
Expand Down
6 changes: 3 additions & 3 deletions src/data_lifecycle/data_conf.cc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

extern DbConfig *dbConfig;
extern char cache_file[256];
char agent_file[256] = "/etc/dtc/agent.xml";
char agent_file[256] = "../conf/agent.xml";

DataConf::DataConf(){
}
Expand Down Expand Up @@ -54,8 +54,8 @@ uint32_t DataConf::Port(){

int DataConf::LoadConfig(int argc, char *argv[]){
int c;
strcpy(table_file, "/etc/dtc/dtc-conf-1000.yaml");
strcpy(cache_file, "/etc/dtc/dtc-conf-1000.yaml");
strcpy(table_file, "../conf/dtc-conf-1000.yaml");
strcpy(cache_file, "../conf/dtc-conf-1000.yaml");

while ((c = getopt(argc, argv, "df:t:hvV")) != -1) {
switch (c) {
Expand Down
2 changes: 1 addition & 1 deletion src/hwcserver/hwc_global.h
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
#define READER_SLEEP_TIME 500 //500ms
#define READER_RETRY_COUNT 20

#define SYS_CONFIG_FILE "/etc/dtc/hbp.conf"
#define SYS_CONFIG_FILE "../conf/hbp.conf"
/*
* err code
*/
Expand Down
2 changes: 1 addition & 1 deletion src/libs/common/config/dbconfig.cc
Original file line number Diff line number Diff line change
Expand Up @@ -592,7 +592,7 @@ int DbConfig::get_dtc_config(YAML::Node dtc_config, DTCConfig* raw, int i_server
m->role[0].addr = STRDUP(dtc_config["primary"][layer]["real"][i]["addr"].as<string>().c_str());
m->role[0].user = STRDUP(dtc_config["primary"][layer]["real"][i]["user"].as<string>().c_str());
m->role[0].pass = STRDUP(dtc_config["primary"][layer]["real"][i]["pwd"].as<string>().c_str());
m->role[0].optfile = STRDUP("/etc/dtc/my.conf");
m->role[0].optfile = STRDUP("../conf/my.conf");
log4cplus_info("addr:%s,user:%s" , m->role[0].addr , m->role[0].user);

/* master DB settings */
Expand Down
2 changes: 1 addition & 1 deletion src/libs/common/config/parse_cluster_config.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
#include <map>
namespace ClusterConfig
{
#define CLUSTER_CONFIG_FILE "/etc/dtc/clusterconfig.xml"
#define CLUSTER_CONFIG_FILE "../conf/clusterconfig.xml"
struct ClusterNode {
std::string name;
std::string addr;
Expand Down
2 changes: 0 additions & 2 deletions src/libs/common/daemon/daemon.cc
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,6 @@ int load_entry_parameter(int argc, char **argv)

//init_log("dtcd");
log4cplus_info("%s v%s: starting....", stat_project_name, version);
//strcpy(d_table_file, "/etc/dtc/dtc.yaml");
//strcpy(d_cache_file, "/etc/dtc/dtc.yaml");
g_dtc_config = new DTCConfig;
//load config file and copy it to ../stat
if (g_dtc_config->load_yaml_file(d_table_file, true) == -1)
Expand Down
4 changes: 2 additions & 2 deletions src/libs/common/data_connector_ask_chain.cc
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,7 @@ int DataConnectorAskChain::migrate_db(DTCJobOperation *job)

// save new table.yaml as table%d.conf
char tableName[64];
snprintf(tableName, 64, "/etc/dtc/dtc%d.conf", tableNo);
snprintf(tableName, 64, "../conf/dtc%d.conf", tableNo);
log4cplus_debug("table.yaml: %s", tableName);
if (dbConfig[1]->cfgObj->Dump(tableName, true) != 0) {
log4cplus_error("save table.yaml as table2.conf error");
Expand Down Expand Up @@ -497,7 +497,7 @@ int DataConnectorAskChain::switch_db(DTCJobOperation *job)
dbConfig[1]->destory();
dbConfig[1] = NULL;
// write conf file
dbConfig[0]->cfgObj->Dump("/etc/dtc/dtc.yaml", false);
dbConfig[0]->cfgObj->Dump("../conf/dtc.yaml", false);
Cleanup();

return 0;
Expand Down
2 changes: 1 addition & 1 deletion src/libs/common/dtc_global.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

#define TABLE_CONF_NAME "../conf/dtc.yaml"
#define CACHE_CONF_NAME "../conf/dtc.yaml"
#define ALARM_CONF_FILE "/etc/dtc/dtcalarm.conf"
#define ALARM_CONF_FILE "../conf/dtcalarm.conf"

class DTCGlobal : private noncopyable {
public:
Expand Down
2 changes: 1 addition & 1 deletion src/libs/common/log/log.h
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ using namespace log4cplus::helpers;

static Logger logger = Logger::getRoot();

#define LOG4CPLUS_CONF_FILE "/etc/dtc/log4cplus.conf"
#define LOG4CPLUS_CONF_FILE "../log4cplus.conf"

/*********************************************
**TRACE:调试应用的详细步骤
Expand Down
2 changes: 1 addition & 1 deletion src/libs/common/table/table_def_manager.cc
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ bool TableDefinitionManager::renew_cur_table_def()

bool TableDefinitionManager::save_new_table_conf()
{
_save_dbconfig->cfgObj->Dump("/etc/dtc/dtc.yaml", false);
_save_dbconfig->cfgObj->Dump("../conf/dtc.yaml", false);
_save_dbconfig->destory();
_save_dbconfig = NULL;
return true;
Expand Down
2 changes: 1 addition & 1 deletion src/libs/net/config.cc
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ int CConfig::ParseBufferedConfig(char *buf, const char *fn, const char *defsec,
{
/*add by foryzhou write config file to stat directory,ignore error*/
char bak_config[1024];
snprintf(bak_config,sizeof(bak_config),"cp %s /etc/dtc/stat/",fn);
snprintf(bak_config,sizeof(bak_config),"cp %s ../conf/stat/",fn);
system(bak_config);
}
return ret_code;
Expand Down
2 changes: 1 addition & 1 deletion src/libs/net/log.h
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ using namespace log4cplus::helpers;

static Logger logger = Logger::getRoot();

#define LOG4CPLUS_CONF_FILE "/etc/dtc/log4cplus.conf"
#define LOG4CPLUS_CONF_FILE "../conf/log4cplus.conf"

/*********************************************
**TRACE:调试应用的详细步骤
Expand Down
2 changes: 1 addition & 1 deletion src/rule/log.h
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ using namespace log4cplus::helpers;

static Logger logger = Logger::getRoot();

#define LOG4CPLUS_CONF_FILE "/etc/dtc/log4cplus.conf"
#define LOG4CPLUS_CONF_FILE "../conf/log4cplus.conf"

/*********************************************
**TRACE:调试应用的详细步骤
Expand Down
2 changes: 1 addition & 1 deletion src/rule/re_load.cc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
#include <iostream>
#include "re_comm.h"

std::string conf_file = "/etc/dtc/dtc.yaml";
std::string conf_file = "../conf/dtc.yaml";

using namespace hsql;
hsql::SQLParserResult rule_ast;
Expand Down
2 changes: 1 addition & 1 deletion src/utils/conf-gen-utils.cc
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#include <map>

using namespace std;
#define ROOT_PATH "/etc/dtc/"
#define ROOT_PATH "../conf/"

char conf_dir[256] = {0};
std::map<std::string, std::vector<YAML::Node>> dbmap;
Expand Down
2 changes: 1 addition & 1 deletion src/utils/log.h
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ using namespace log4cplus::helpers;

static Logger logger = Logger::getRoot();

#define LOG4CPLUS_CONF_FILE "/etc/dtc/log4cplus.conf"
#define LOG4CPLUS_CONF_FILE "../conf/log4cplus.conf"

/*********************************************
**TRACE:调试应用的详细步骤
Expand Down

0 comments on commit 8f7c63d

Please sign in to comment.