[feat] tools-v2: add bs check server #2463

pengpengSir · 2023-05-08T12:19:08Z

What problem does this PR solve?

Issue Number: #2355

Problem Summary: support bs check server command in curve tool

What is changed and how it works?

What's Changed:
add tools-v2/pkg/cli/command/curvebs/check/server/server.go
add tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go
modify tools-v2/internal/utils/row.go
modify tools-v2/pkg/config/bs.go
modify tools-v2/pkg/cli/command/curvebs/check/check.go

How it Works:

Usage:  curve bs check server [flags]

check all copysets infomation on server

Flags:
  -c, --conf string           config file (default is $HOME/.curve/curve.yaml or /etc/curve/curve.yaml)
  -f, --format string         output format (json|plain) (default "plain")
  -h, --help                  print help
      --mdsaddr string        mds address, should be like 127.0.0.1:6700,127.0.0.1:6701,127.0.0.1:6702
      --port int              port
      --rpcretrytimes int32   rpc retry times (default 1)
      --rpctimeout duration   rpc timeout (default 10s)
      --serverid int          server id
      --serverip string       server ip
      --showerror             display all errors in command
      --verbose               show some log

Examples:
$ curve bs check server

command result

+--------+-----------+-------+-------------------------+-------------------------+
| SERVER |    IP     | TOTAL | UNHEALTHY COPYSET COUNT | UNHEALTHY COPYSET RATIO |
+--------+-----------+-------+-------------------------+-------------------------+
| 1      | 127.0.0.1 | 100   | 0                       | 0                       |
+--------+-----------+-------+-------------------------+-------------------------+

Side effects(Breaking backward compatibility? Performance regression?):

Check List

Relevant documentation/comments is changed or added
I acknowledge that all my contributions will be made under the project's license

Cyber-SiKu · 2023-05-09T06:44:56Z

when i try to build this, get such error:

$ go build -gcflags='all=-N -l'  -o sbin/curve ./cmd/curve/main.go
# github.com/opencurve/curve/tools-v2/pkg/cli/command/curvebs/check/server
pkg/cli/command/curvebs/check/server/get_copyset_ids.go:75:9: undefined: config.AddBsChunkServerIDFlag
pkg/cli/command/curvebs/check/server/get_copyset_ids.go:76:9: undefined: config.AddBsHostIpFlag
pkg/cli/command/curvebs/check/server/get_copyset_ids.go:85:22: assignment mismatch: 2 variables but config.GetBsFlagUint32 returns 1 value
pkg/cli/command/curvebs/check/server/get_copyset_ids.go:87:13: assignment mismatch: 2 variables but config.GetBsFlagUint32 returns 1 value
pkg/cli/command/curvebs/check/server/server.go:107:17: assignment mismatch: 2 variables but config.GetBsFlagUint32 returns 1 value
pkg/cli/command/curvebs/check/server/server.go:109:13: assignment mismatch: 2 variables but config.GetBsFlagUint32 returns 1 value
pkg/cli/command/curvebs/check/server/server.go:181:10: undefined: config.AddBsChunkServerIDFlag
pkg/cli/command/curvebs/check/server/server.go:182:10: undefined: config.AddBsHostIpFlag

tools-v2/pkg/cli/command/curvebs/check/server/server.go

Cyber-SiKu · 2023-05-10T12:08:29Z

tools-v2/pkg/cli/command/curvebs/check/server/server.go

+
+type CheckServerRpc struct {
+	Info           *basecmd.Rpc
+	Request        *topology.ListChunkServerRequest


add listchunkserver in list.
However, other pr is already doing this part of the content, and I have explained the relevant amendments.

Or you can communicate with him and ask him to reserve a corresponding interface so that you can continue.

Already talked to him, waiting for his response.

Cyber-SiKu · 2023-05-10T12:11:07Z

tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go

+	config.AddBsHostIpFlag(cCmd.Cmd)
+	config.AddBsPortFlag(cCmd.Cmd)


chunkaddr is better?

Since hostip and port are separated in topology.GetCopySetsInChunkServerRequest, so i think original version is better.

Cyber-SiKu · 2023-05-16T07:26:09Z

tools-v2/README.md

@@ -1291,6 +1292,27 @@ Output:
 +------------+-----------+--------+--------+--------+---------+
 ```

+####  check server


Complement the comparison of old and new tools

Comparison of old and new commands

curve bs

Cyber-SiKu · 2023-05-16T07:27:09Z

tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go

+	hostip := config.GetBsFlagString(cCmd.Cmd, config.CURVEBS_HOST_IP)
+	port := config.GetBsFlagUint32(cCmd.Cmd, config.CURVEBS_PORT)
+
+	// fmt.Printf("%d %s %d", chunkserverId, hostip, port)


@pengpengSir here

Cyber-SiKu · 2023-05-16T07:27:28Z

tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go

+}
+
+// 根据ChunkServerID, HostIP, Port返回 copysetid2poolid


please use english

.gitmodules

WORKSPACE

Xinlong-Chen · 2023-05-20T03:57:43Z

tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go

+	return output.FinalCmdOutputPlain(&cCmd.FinalCurveCmd)
+}
+
+func (cCmd *CopysetidsCommand) RunCommand(cmd *cobra.Command, args []string) error {


we can set Cmd.Error = err, to let client use this error.

tools-v2/pkg/cli/command/curvebs/check/server/server.go

Xinlong-Chen · 2023-05-20T04:15:41Z

tools-v2/pkg/cli/command/curvebs/check/server/server.go

+
+		peerAddr := fmt.Sprintf("%s:%d", hostip, port)
+		for copysetid, poolid := range *copysetids2poolids {
+			pCmd.Cmd.ResetFlags()


can reuse flag?

Got error when trying to reuse these flags

what error?

Got error when trying to reuse these flags

Because I try to use GetCopysetStatus in GetCopysetStatusCommand. Flags I passed to GetCopysetStatusCommand accumulate in its Rpc list, so I think the possible way to reuse flags is GetCopysetStatusCommand offers a method to reset Rpc list.

Signed-off-by: pengpengSir <[email protected]>

opencurveadmin · 2023-05-22T07:30:27Z

@Cyber-SiKu PTAL

Xinlong-Chen

please take care of forementioned problems...
Without more problem for me now!

Cyber-SiKu · 2023-05-23T06:29:23Z

tools-v2/README.md

@@ -1405,6 +1428,7 @@ Output:
 | curve_ops_tool check-copyset         | curve bs check copyset         |
 | curve_ops_tool client-status         | curve bs status client         |
 | curve_ops_tool check-operator        | curve bs check operator        |
+| curve_ops_tool check-server          | curve bs check server          |


Delete line 1444, and please don't delete the blank line at the end.

Cyber-SiKu · 2023-05-23T06:38:00Z

tools-v2/pkg/config/bs.go

@@ -501,6 +513,38 @@ func AddBSCSUnhealthyOptionFlag(cmd *cobra.Command) {
 	AddBsBoolOptionFlag(cmd, CURVEBS_CS_UNHEALTHY, "unhealthy")
 }

+func AddBsServerIdFlag(cmd *cobra.Command) {


Suggested change

func AddBsServerIdFlag(cmd *cobra.Command) {

func AddBsServerIdOptionFlag(cmd *cobra.Command) {

Cyber-SiKu · 2023-05-23T06:38:10Z

tools-v2/pkg/config/bs.go

+	AddBSUint32OptionFlag(cmd, CURVEBS_SERVER_ID, "server id")
+}
+
+func AddBsServerIpFlag(cmd *cobra.Command) {


Suggested change

func AddBsServerIpFlag(cmd *cobra.Command) {

func AddBsServerIpOptionFlag(cmd *cobra.Command) {

Cyber-SiKu · 2023-05-23T06:38:18Z

tools-v2/pkg/config/bs.go

+	AddBsStringOptionFlag(cmd, CURVEBS_SERVER_IP, "server ip")
+}
+
+func AddBsPortFlag(cmd *cobra.Command) {


Suggested change

func AddBsPortFlag(cmd *cobra.Command) {

func AddBsPortOptionFlag(cmd *cobra.Command) {

Cyber-SiKu · 2023-05-23T06:38:31Z

tools-v2/pkg/config/bs.go

+	AddBsStringOptionFlag(cmd, CURVEBS_CHUNKSERVER_ID, "chunk server id")
+}
+
+func AddBsHostIpFlag(cmd *cobra.Command) {


Suggested change

func AddBsHostIpFlag(cmd *cobra.Command) {

func AddBsHostIpOptionFlag(cmd *cobra.Command) {

Cyber-SiKu · 2023-05-23T06:41:28Z

tools-v2/pkg/config/bs.go

+	CURVEBS_SERVER_ID               = "serverid"
+	VIPER_CURVEBS_SERVER_ID         = "curvebs.serverid"
+	CURVEBS_SERVER_IP               = "serverip"
+	VIPER_CURVEBS_SERVER_IP         = "curvebs.serverip"
+	CURVEBS_HOST_IP                 = "hostip"
+	VIPER_CURVEBS_HOST_IP           = "curvebs.hostip"
+	CURVEBS_PORT                    = "port"
+	VIPER_CURVEBS_PORT              = "curvebs.port"


If some flags are optional, please give default values.

Cyber-SiKu · 2023-05-23T06:48:50Z

tools-v2/internal/utils/row.go

+	ROW_UNHEALTHY_COPYSET_RATIO = "unhealthy coyset ratio"
+	ROW_UNHEALTHY_COPYSET_COUNT = "unhealthy copyset count"


format
Alignment, and I think, only keep ROW_UNHEALTHY_COPYSET, the following output is also good:
|23(10%)|

Cyber-SiKu · 2023-05-23T06:51:31Z

tools-v2/pkg/cli/command/curvebs/check/server/server.go

@@ -0,0 +1,244 @@
+/*
+ *  Copyright (c) 2022 NetEase Inc.


Suggested change

* Copyright (c) 2022 NetEase Inc.

* Copyright (c) 2023 NetEase Inc.

Cyber-SiKu · 2023-05-23T06:54:00Z

tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go

+	config.AddRpcRetryTimesFlag(cCmd.Cmd)
+	config.AddRpcTimeoutFlag(cCmd.Cmd)
+	config.AddBsChunkServerIDOptionFlag(cCmd.Cmd)
+	config.AddBsHostIpFlag(cCmd.Cmd)


Both hostip and serverip are essentially ip, can these two be called ip? Is it convenient to pass it on?

Cyber-SiKu · 2023-05-23T06:55:20Z

tools-v2/pkg/cli/command/curvebs/check/server/server.go

+
+type ServerCommand struct {
+	basecmd.FinalCurveCmd
+	Rpc      *listchunkserver.ListChunkServerRpc


Such an rpc can be independently used to make a command and then return the result so that other commands can reuse the rpc.

Signed-off-by: pengpengSir <[email protected]>

Cyber-SiKu · 2023-05-24T06:54:33Z

tools-v2/pkg/cli/command/curvebs/check/server/server.go

+	var rpc *listchunkserver.ListChunkServerRpc
+	if sCmd.ServerIP != "" {
+		rpc = &listchunkserver.ListChunkServerRpc{
+			Request: &topology.ListChunkServerRequest{
+				Ip:   &sCmd.ServerIP,
+				Port: &sCmd.Port,
+			},
+			Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),
+		}
+	} else {
+		rpc = &listchunkserver.ListChunkServerRpc{
+			Request: &topology.ListChunkServerRequest{
+				ServerID: &sCmd.ServerID,
+			},
+			Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),
+		}
+	}
+


Here you can separate the list chunkserver rpc into a separate command, and then call this command. For example fileinfo.go
These things can be processed in the command.

Signed-off-by: pengpengSir <[email protected]>

Cyber-SiKu · 2023-05-26T04:13:41Z

tools-v2/pkg/cli/command/curvebs/check/server/server.go

+
+		for _, status := range copysetid2Status {
+			total++
+			if status.String() == "COPYSET_OP_STATUS_SUCCESS" {


use constant

Signed-off-by: pengpengSir <[email protected]>

zhanghuidinah · 2023-05-31T08:26:29Z

cicheck

Cyber-SiKu · 2023-06-01T07:26:40Z

cicheck

Cyber-SiKu · 2023-06-01T07:35:24Z

tools-v2/pkg/cli/command/curvebs/check/server/get_chunk_info.go

+	if ip != "" {
+		rpc := &GetChunkServerInfosRpc{
+			Request: &topology.ListChunkServerRequest{
+				Ip:   &ip,
+				Port: &port,
+			},
+			Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),
+		}
+		gCmd.Rpc = rpc
+	} else {
+		rpc := &GetChunkServerInfosRpc{
+			Request: &topology.ListChunkServerRequest{
+				ServerID: &serverid,
+			},
+			Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),
+		}
+		gCmd.Rpc = rpc
+	}


Suggested change

if ip != "" {

rpc := &GetChunkServerInfosRpc{

Request: &topology.ListChunkServerRequest{

Ip: &ip,

Port: &port,

},

Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),

}

gCmd.Rpc = rpc

} else {

rpc := &GetChunkServerInfosRpc{

Request: &topology.ListChunkServerRequest{

ServerID: &serverid,

},

Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),

}

gCmd.Rpc = rpc

}

gCmd.Rpc = &GetChunkServerInfosRpc{

Request: &topology.ListChunkServerRequest{},

Info: basecmd.NewRpc(mdsAddrs, timeout, retrytimes, "ListChunkServer"),

}

if ip != "" {

gCmd.Rpc.Request.Ip = &ip

gCmd.Rpc.Request.Port = &port

} else {

gCmd.Rpc.Request.ServerID = &serverid

}

Cyber-SiKu · 2023-06-01T07:49:01Z

ip && total is wrong

montaguelhz · 2023-06-07T04:29:12Z

tools-v2/pkg/cli/command/curvebs/check/server/get_copyset_ids.go

+func (cCmd *CopysetidsCommand) Init(cmd *cobra.Command, args []string) error {
+	timeout := config.GetFlagDuration(cCmd.Cmd, config.RPCTIMEOUT)
+	retrytimes := config.GetFlagInt32(cCmd.Cmd, config.RPCRETRYTIMES)
+	mdsAddrs, _ := config.GetBsMdsAddrSlice(cCmd.Cmd)


why not check the error

SeanHai · 2023-07-09T03:02:17Z

This pr has not progressed for a while, did you encounter any difficulties?

Signed-off-by: caoxianfei1 <[email protected]>

caoxianfei1 · 2023-07-30T11:14:55Z

Clost the pt because the feature has been merged.

zhanghuidinah requested a review from Cyber-SiKu May 9, 2023 02:23

Cyber-SiKu mentioned this pull request May 10, 2023

feat: [curve/toos-v2] add list chunkserver #2386

Merged

2 tasks

Cyber-SiKu reviewed May 10, 2023

View reviewed changes

zhanghuidinah added the Developer Activities label May 15, 2023

Cyber-SiKu reviewed May 16, 2023

View reviewed changes

opencurveadmin requested review from Cyber-SiKu and Xinlong-Chen May 19, 2023 06:46

pengpengSir force-pushed the master branch from 6105640 to 10dfcb7 Compare May 20, 2023 03:37

Xinlong-Chen reviewed May 20, 2023

View reviewed changes

fix bs check server

ddedf5f

Signed-off-by: pengpengSir <[email protected]>

pengpengSir force-pushed the master branch from d2d5826 to ddedf5f Compare May 20, 2023 12:27

pengpengSir requested a review from Xinlong-Chen May 22, 2023 07:10

Xinlong-Chen reviewed May 23, 2023

View reviewed changes

Cyber-SiKu reviewed May 23, 2023

View reviewed changes

fix bs check server

c607530

Signed-off-by: pengpengSir <[email protected]>

Cyber-SiKu reviewed May 24, 2023

View reviewed changes

seperate command

8427f05

Signed-off-by: pengpengSir <[email protected]>

pengpengSir force-pushed the master branch from a3f0c95 to 8427f05 Compare May 24, 2023 12:40

Cyber-SiKu reviewed May 26, 2023

View reviewed changes

Update server.go

35e5f3b

Signed-off-by: pengpengSir <[email protected]>

Cyber-SiKu reviewed Jun 1, 2023

View reviewed changes

montaguelhz reviewed Jun 7, 2023

View reviewed changes

Cyber-SiKu added the pending label Jul 21, 2023

caoxianfei1 pushed a commit to caoxianfei1/curve that referenced this pull request Jul 23, 2023

fix the pr opencurve#2463

f886190

Signed-off-by: caoxianfei1 <[email protected]>

caoxianfei1 pushed a commit to caoxianfei1/curve that referenced this pull request Jul 23, 2023

fix the pr opencurve#2463

9c62101

Signed-off-by: caoxianfei1 <[email protected]>

caoxianfei1 pushed a commit to caoxianfei1/curve that referenced this pull request Jul 23, 2023

fix the pr opencurve#2463

db1b30e

Signed-off-by: caoxianfei1 <[email protected]>

caoxianfei1 pushed a commit to caoxianfei1/curve that referenced this pull request Jul 23, 2023

fix the pr opencurve#2463

30dbc25

Signed-off-by: caoxianfei1 <[email protected]>

This was referenced Jul 23, 2023

[tools-v2] add bs check server based on pr2463 #2643

Closed

[tools-v2] add bs check server based on pr2463 #2646

Merged

caoxianfei1 closed this Jul 30, 2023

		config.AddBsHostIpFlag(cCmd.Cmd)
		config.AddBsPortFlag(cCmd.Cmd)

		}

		// 根据ChunkServerID, HostIP, Port返回 copysetid2poolid

	func AddBsServerIdFlag(cmd *cobra.Command) {
	func AddBsServerIdOptionFlag(cmd *cobra.Command) {

	func AddBsServerIpFlag(cmd *cobra.Command) {
	func AddBsServerIpOptionFlag(cmd *cobra.Command) {

	func AddBsPortFlag(cmd *cobra.Command) {
	func AddBsPortOptionFlag(cmd *cobra.Command) {

	func AddBsHostIpFlag(cmd *cobra.Command) {
	func AddBsHostIpOptionFlag(cmd *cobra.Command) {

		ROW_UNHEALTHY_COPYSET_RATIO = "unhealthy coyset ratio"
		ROW_UNHEALTHY_COPYSET_COUNT = "unhealthy copyset count"

	* Copyright (c) 2022 NetEase Inc.
	* Copyright (c) 2023 NetEase Inc.

[feat] tools-v2: add bs check server #2463

[feat] tools-v2: add bs check server #2463

Conversation

pengpengSir commented May 8, 2023

What problem does this PR solve?

What is changed and how it works?

Check List

Cyber-SiKu commented May 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Comparison of old and new commands

curve bs

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Xinlong-Chen May 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

opencurveadmin commented May 22, 2023

Xinlong-Chen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhanghuidinah commented May 31, 2023

Cyber-SiKu commented Jun 1, 2023

Choose a reason for hiding this comment

Cyber-SiKu commented Jun 1, 2023 • edited Loading

Choose a reason for hiding this comment

SeanHai commented Jul 9, 2023

caoxianfei1 commented Jul 30, 2023

Xinlong-Chen May 23, 2023 •

edited

Loading

Cyber-SiKu commented Jun 1, 2023 •

edited

Loading