Abel'Blog

我干了什么?究竟拿了时间换了什么?

0%

go-protobuf

发表于 更新于 分类于 ，

简介

protobuf是比较好的一种序列化的工具，这里使用golang来试用一下protobuf

protobuf模块

参考文档

准备工具

安装命令

安装效果

环境变量配置

定义protobuf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
syntax = "proto3";
package example;

message Person {
  string name = 1;
  int32 id = 2;  // Unique ID number for this person.
  string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
  }

  repeated PhoneNumber phones = 4;

}

// Our address book file is just one of these.
message AddressBook {
  repeated Person people = 1;
}

生成go文件

1
protoc.exe -I=D:/gopath/src/IDL --go_out=./ D:/gopath/src/IDL/addressbook.proto

编写序列化、反序列化代码

参考代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
package main

import (
    "fmt"
    "github.com/golang/protobuf/proto"

    pb "octopus.com/test/test_protobuf/example"
)

func main() {

    p := &pb.Person{
        Id:    1234,
        Name:  "John Doe",
        Email: "jdoe@example.com",
        Phones: []*pb.Person_PhoneNumber{
            {Number: "555-4321", Type: pb.Person_HOME},
        },
    }

    fmt.Println(p)

    out, err := proto.Marshal(p)
    if err != nil {
        fmt.Println(err)
        return
    }

    fmt.Println(out)

}

输出：

1
2
name:"John Doe" id:1234 email:"jdoe@example.com" phones:<number:"555-4321" type:HOME > 
[10 8 74 111 104 110 32 68 111 101 16 210 9 26 16 106 100 111 101 64 101 120 97 109 112 108 101 46 99 111 109 34 12 10 8 53 53 53 45 52 51 50 49 16 1]

golang常用的内存处理方式

读取数据包的实例

写入数据实例

1
2
3
4
5
buf := bytes.NewBuffer(nil)
var id uint32 = 2
var _type uint16 =3
binary.Write(buf, binary.LittleEndian, &id)
binary.Write(buf, binary.LittleEndian, &_type)

从内存块中读取内容

1
2
3
4
5
6
7
8
9
10
11

var data []byte
//... 赋值数据

buf := bytes.NewBuffer(data)

var id uint32
var _type uint16
binary.Read(buf, binary.LittleEndian, &id)
binary.Read(buf, binary.LittleEndian, &_type)

PutUint32的源码实现：

1
2
3
4
5
6
7
func (bigEndian) PutUint32(b []byte, v uint32) {
    _ = b[3] // early bounds check to guarantee safety of writes below
    b[0] = byte(v >> 24)
    b[1] = byte(v >> 16)
    b[2] = byte(v >> 8)
    b[3] = byte(v)
}

protobuf冲突

问题是多个库都需要使用，而且这个库里有重复定义的protobuf，这个时候可以通过下面的方法将这个问题忽略掉。

1
2
WARNING: proto: file "Tron.proto" has a name conflict over protocol.ChainInventory
See https://protobuf.dev/reference/go/faq#namespace-conflict
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
// conflictPolicy configures the policy for handling registration conflicts.
//
// It can be over-written at compile time with a linker-initialized variable:
//
//	go build -ldflags "-X google.golang.org/protobuf/reflect/protoregistry.conflictPolicy=warn"
//
// It can be over-written at program execution with an environment variable:
//
//	GOLANG_PROTOBUF_REGISTRATION_CONFLICT=warn ./main
//
// Neither of the above are covered by the compatibility promise and
// may be removed in a future release of this module.
var conflictPolicy = "panic" // "panic" | "warn" | "ignore"

// ignoreConflict reports whether to ignore a registration conflict
// given the descriptor being registered and the error.
// It is a variable so that the behavior is easily overridden in another file.
var ignoreConflict = func(d protoreflect.Descriptor, err error) bool {
	const env = "GOLANG_PROTOBUF_REGISTRATION_CONFLICT"
	const faq = "https://protobuf.dev/reference/go/faq#namespace-conflict"
	policy := conflictPolicy
	if v := os.Getenv(env); v != "" {
		policy = v
	}
	switch policy {
	case "panic":
		panic(fmt.Sprintf("%v\nSee %v\n", err, faq))
	case "warn":
		fmt.Fprintf(os.Stderr, "WARNING: %v\nSee %v\n\n", err, faq)
		return true
	case "ignore":
		return true

编译的时候去掉这个选项

1
MYGOFLAGS=-ldflags "-s -w -X google.golang.org/protobuf/reflect/protoregistry.conflictPolicy=ignore" # Add your custom build flags here

调试器里可以设置一个环境变量来处理

1
2
3
4

"env": {
    "GOLANG_PROTOBUF_REGISTRATION_CONFLICT": "ignore"
},

参考

- [1] golang标准库binary学习