Skip to content

Commit

Permalink
Merge branch 'main' of github.com:linkall-labs/vanus into segment
Browse files Browse the repository at this point in the history
Signed-off-by: jyjiangkai <[email protected]>
  • Loading branch information
hwjiangkai committed Feb 6, 2023
2 parents 5b3b241 + 1157347 commit 2bf7967
Show file tree
Hide file tree
Showing 382 changed files with 19,251 additions and 8,512 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ jobs:
startsWith(github.event.pull_request.title, 'fix') ||
startsWith(github.event.pull_request.title, 'feat') ||
startsWith(github.event.pull_request.title, 'refactor') ||
startsWith(github.event.pull_request.title, 'perf')
startsWith(github.event.pull_request.title, 'perf') ||
startsWith(github.event.pull_request.title, 'test')
runs-on: ${{ matrix.os }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Expand Down
5 changes: 4 additions & 1 deletion .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,9 @@ issues:
- source: "strconv|make|len|math"
linters:
- gomnd
- path: "action"
linters:
- dupl
- path: "convert.go"
linters:
- dupl
Expand All @@ -412,7 +415,7 @@ issues:
linters:
- gosec
- gomnd
- path: "action.go"
- path: "init.go"
linters:
- gochecknoinits
- path: "^vsctl"
Expand Down
9 changes: 3 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ resources to make it easier to get your contribution accepted.
## Before you get started

### Sign the CLA
Click [here](https://cla-assistant.io/linkall-labs/) to sign the CLA, and click the `Sign in with GitHub to agree` button to sign.
Click [here](https://cla-assistant.io/linkall-labs/vanus) to sign the CLA, and click the `Sign in with GitHub to agree` button to sign.

What is [CLA](https://en.wikipedia.org/wiki/Contributor_License_Agreement).

Expand All @@ -39,7 +39,7 @@ If you want to contribute to the codebase of Vanus, it's better to set up a deve
1. Install Go version 1.17 or above. Refer to [How to Write Go Code](https://go.dev/doc/code) for more information.
2. Install [minikube](https://minikube.sigs.k8s.io/docs/start/)(Users from China mainland may have network issues that need to be addressed for using minikube).
Vanus is a k8s based project.
3. Learning [QuickStart](https://github.com/linkall-labs/docs/blob/main/vanus/quick-start.md) how to deploy the Vanus to k8s. Deploying a specified component to minikube for testing.
3. Learning [QuickStart](https://docs.linkall.com/getting-started/quick-start) how to deploy the Vanus to k8s. Deploying a specified component to minikube for testing.
Click [here](#) for details.

## Issues
Expand All @@ -51,7 +51,7 @@ assigned to someone.

#### First Contribution
You can start by finding existing issues with the
[good-first-issue](https://github.com/linkall-labs/vanus/issues?q=is%3Aopen+is%3Aissue+label%3Agood-first-issue) label.
[good-first-issue](https://github.com/linkall-labs/vanus/issues?q=is%3Aopen+is%3Aissue+label%3A"good+first+issue") label.
These issues are well suited for a new contributor.

#### Further Contributions
Expand Down Expand Up @@ -113,6 +113,3 @@ There are some automated tasks to ensure each PR's reach high quality in Vanus.
### Code Review
The Code Review can be started until all checking were passed. Each PR will automatically assign reviewers. After getting
at least one `Approve` from reviewers, the PR can be merged.

## Reward
Once your PR has been merged, you become a Vanus Contributor. Thanks for your contribution! please fill the [form](#) to get you reward
37 changes: 2 additions & 35 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -75,38 +75,5 @@ docker-build-timer:
build-timer:
$(GO_BUILD) -o bin/timer cmd/timer/main.go

controller-start:
go run ${VANUS_ROOT}/cmd/controller/main.go

build-ctrl-bin:
$(GO_BUILD) -o bin/ctrl cmd/controller/main.go

build-gw-util:
go build -o bin/gw-util test/gateway/main.go

build-e2e:
go build -o bin/e2e test/e2e/quick-start/main.go

build-destruct:
go build -o bin/destruct test/e2e/destruct/main.go

controller-start:
go run ${VANUS_ROOT}/cmd/controller/${module}/main.go

controller-api-test:
grpcui --import-path=${VSPROTO_ROOT}/include \
--import-path=${VSPROTO_ROOT}/proto \
--plaintext \
--proto=controller.proto 127.0.0.1:2048

store-start:
go run ${VANUS_ROOT}/cmd/store/main.go

store-api-test:
grpcui --import-path=${VSPROTO_ROOT}/include \
--import-path=${VSPROTO_ROOT}/proto \
--plaintext \
--proto=segment.proto 127.0.0.1:11831

trigger-start:
go run ${VANUS_ROOT}/cmd/trigger/main.go
docker-push-aio:
docker buildx build --platform ${DOCKER_PLATFORM} -t ${DOCKER_REPO}/all-in-one:${IMAGE_TAG} -f build/all-in-one/Dockerfile . --push
95 changes: 73 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,79 @@
# Vanus

<div align="center">
<h1>Vanus</h1>
<!--
<img src="https://user-images.githubusercontent.capabilitiesom/68597908/206148625-43f14f58-f3c0-4042-82a0-9f9421c270fa.png" width="288" >
-->
<p>
Vanus is an open-source message queue with built-in event processing capabilities.
<!--
<br>
It helps users build event pipelines between SaaS, cloud services, and cloud functions in minutes.
-->
</p>

[![stars](https://img.shields.io/github/stars/linkall-labs/vanus.svg?style=flat&logo=github&colorB=blueviolet&label=stars)](https://github.com/linkall-labs/vanus)
[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](https://github.com/linkall-labs/vanus/blob/main/LICENSE)
[![Language](https://img.shields.io/badge/Language-Go-blue.svg)](https://golang.org/)
[![codecov](https://codecov.io/gh/linkall-labs/vanus/branch/main/graph/badge.svg?token=RSXSIMEY4V)](https://codecov.io/gh/linkall-labs/vanus)

## What is Vanus

Vanus is a Serverless event platform for easily building Event-Driven Architecture (EDA) applications.
It provides enterprises and organizations an innovative platform for collecting, storing, distributing, and processing events at scale.

## Quick Start

See [quick-start](https://docs.linkall.com/getting-started/quick-start)
[![Language](https://img.shields.io/github/go-mod/go-version/linkall-labs/vanus?logo=go)](https://golang.org/)
[![playground](https://img.shields.io/badge/Playground-Try%20it%20%20free-red)](https://play.linkall.com)
[![playground](https://img.shields.io/badge/Docs-online-green)](https://linkall.com/docs)


<p>
<a href="https://join.slack.com/t/vanusworkspace/shared_invite/zt-1jilbbfo2-NxiFG0VOo8ABGCCNaeNfcA"><img src="https://img.shields.io/badge/slack-join-3CC798?style=social&logo=slack" height=23></a>
&nbsp;
<a href="https://twitter.com/Vanus_dev"><img src="https://img.shields.io/badge/-Twitter-red?style=social&logo=twitter" height=23></a>
<!-- <a href="https://twitter.com/Vanus_dev"><img src="https://img.shields.io/twitter/follow/vanus_dev?style=social" height=23></a> -->
&nbsp;
<a href="https://www.youtube.com/channel/UC7rd6IgjfNYTOXf2FerFNyA"><img src="https://img.shields.io/badge/-Youtube-red?style=social&logo=youtube" height=23></a>
<!-- <a href="https://www.youtube.com/channel/UC7rd6IgjfNYTOXf2FerFNyA"><img src="https://img.shields.io/youtube/channel/views/UC7rd6IgjfNYTOXf2FerFNyA?style=social" height=23></a> -->
&nbsp;
</p>

<!--
<h3 align="center">
<a href="https://featbit.gitbook.io/docs/installation">Installation</a>
<span> · </span>
<a href="https://featbit.gitbook.io/">Getting Started</a>
<span> · </span>
<a href="https://join.slack.com/t/featbit/shared_invite/zt-1ew5e2vbb-x6Apan1xZOaYMnFzqZkGNQ">Online Support</a>
<span> · </span>
<a href="https://featbit.gitbook.io/">Documentation</a>
<span> · </span>
<a href="https://github.com/featbit/featbit/discussions/categories/announcements">Milestones</a>
</h3>
-->
</div>

--------------------------------------------------

## Introduction

Vanus helps users build event pipelines between SaaS, cloud services, and cloud functions in minutes.

**1. Build the event-driven system**
- Get events from cloud services and SaaS, and deliver them to cloud functions or microservices.
- Synchronize changed data or transfer data to the data lake.
- Obtain events generated by SaaS and send them to other SaaS.

**2. Out-of-the-box event computing capabilities**
- Real-time processing during event transmissions, such as filtering and transformation.
- Natively support the CloudEvents specification, and can directly send events to workloads that support CloudEvent.

**3. 100% open source, Super easy to use**
- One-click deployment, the installation is completed within 1 minute, and developers without MQ experience can also use it.
- Message queues and connectors are 100% open source, a one-stop open-source solution.

## Getting Started

You can install Vanus with a single command within 1 minute. Check out our [website](https://linkall.com) for detailed information.

```shell
kubectl apply -f https://download.linkall.com/all-in-one/v0.5.4.yml
```

We also provide an [interactive Kubernetes environment](https://play.linkall.com) to simply deploy and try Vanus in your browser.

## Community

Expand All @@ -26,17 +88,6 @@ We have a few channels for contact:

See [here](CONTRIBUTING.md) for how to contribute to Vanus.

## Understanding Vanus

### Project Layout

This Project follows [golang-standards/project-layout](https://github.com/golang-standards/project-layout), see the
project for understanding well vanus' codebase.

### Architecture

![architecture](docs/architecture.jpg)

## License

Vanus is under the Apache 2.0 license. See the [LICENSE](LICENSE) file for details.
33 changes: 33 additions & 0 deletions build/all-in-one/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
FROM --platform=$BUILDPLATFORM golang:1.18 as builder
WORKDIR /workspace

COPY . .
RUN go mod download

ARG TARGETOS
ARG TARGETARCH
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH make build-controller
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH make build-store
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH make build-trigger
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH make build-timer
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH make build-gateway

FROM centos:latest
WORKDIR /vanus

ARG VERSION
COPY --from=builder /workspace/bin/controller bin/controller
COPY --from=builder /workspace/bin/store bin/store
COPY --from=builder /workspace/bin/trigger bin/trigger
COPY --from=builder /workspace/bin/timer bin/timer
COPY --from=builder /workspace/bin/gateway bin/gateway
COPY --from=builder /workspace/build/all-in-one/config config
COPY --from=builder /workspace/build/all-in-one/run.sh run.sh

RUN chmod a+x /vanus/run.sh
RUN mkdir /vanus/logs

ENV VERSION=$VERSION

ENTRYPOINT ["/vanus/run.sh"]

34 changes: 34 additions & 0 deletions build/all-in-one/config/controller.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: "standalone"
ip: "127.0.0.1"
port: 2048
etcd:
- "127.0.0.1:2379"
data_dir: "/vanus/data/controller"
gateway_endpoint: "127.0.0.1:18080"
segment_capacity: 8388604 # 8MB
topology:
standalone: 127.0.0.1:2048
replicas: 1
metadata:
key_prefix: "/standalone"
embed_etcd:
# relative path to ${data_dir} above
data_dir: "etcd"
listen_client_addr: 127.0.0.1:2379
listen_peer_addr: 127.0.0.1:2380
advertise_client_addr: 127.0.0.1:2379
advertise_peer_addr: 127.0.0.1:2380
heartbeat-interval: 600000
election-timeout: 600000
clusters:
- standalone=http://127.0.0.1:2380
secret_encryption_salt: "encryption_salt"
observability:
metrics:
enable: false
# metrics for prometheus scratch data
port: 2112
tracing:
enable: false
# OpenTelemetry Collector endpoint, https://opentelemetry.io/docs/collector/getting-started/
otel_collector: http://127.0.0.1:4318
12 changes: 12 additions & 0 deletions build/all-in-one/config/gateway.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
port: 8080
controllers:
- "127.0.0.1:2048"
observability:
metrics:
enable: false
# metrics for prometheus scratch data
port: 2113
tracing:
enable: false
# OpenTelemetry Collector endpoint, https://opentelemetry.io/docs/collector/getting-started/
otel_collector: http://127.0.0.1:4318
30 changes: 30 additions & 0 deletions build/all-in-one/config/store.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
port: 11811
ip : 127.0.0.1
controllers:
- 127.0.0.1:2048
volume:
id: 1
dir: /vanus/data/store
capacity: 1073741824 # 1GB
meta_store:
wal:
io:
engine: psync
offset_store:
wal:
io:
engine: psync
raft:
wal:
block_size: 32768
io:
engine: psync
observability:
metrics:
enable: false
# metrics for prometheus scratch data
port: 2114
tracing:
enable: false
# OpenTelemetry Collector endpoint, https://opentelemetry.io/docs/collector/getting-started/
otel_collector: http://127.0.0.1:4318
24 changes: 24 additions & 0 deletions build/all-in-one/config/timer.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: "timer"
ip: "127.0.0.1"
etcd:
- "127.0.0.1:2379"
replicas: 1
metadata:
key_prefix: "/standalone"
leaderelection:
lease_duration: 15
timingwheel:
tick: 1
wheel_size: 32
layers: 4
controllers:
- 127.0.0.1:2048
observability:
metrics:
enable: false
# metrics for prometheus scratch data
port: 2115
tracing:
enable: false
# OpenTelemetry Collector endpoint, https://opentelemetry.io/docs/collector/getting-started/
otel_collector: http://127.0.0.1:4318
14 changes: 14 additions & 0 deletions build/all-in-one/config/trigger.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
port: 2148
ip: "127.0.0.1"
controllers:
- 127.0.0.1:2048
rateLimit: 0
observability:
metrics:
enable: false
# metrics for prometheus scratch data
port: 2116
tracing:
enable: false
# OpenTelemetry Collector endpoint, https://opentelemetry.io/docs/collector/getting-started/
otel_collector: http://127.0.0.1:4318
10 changes: 10 additions & 0 deletions build/all-in-one/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
nohup /vanus/bin/controller --config /vanus/config/controller.yaml > /vanus/logs/controller.log &
nohup /vanus/bin/store --config /vanus/config/store.yaml > /vanus/logs/store.log &
nohup /vanus/bin/trigger --config /vanus/config/trigger.yaml > /vanus/logs/trigger.log &
nohup /vanus/bin/timer --config /vanus/config/timer.yaml > /vanus/logs/timer.log &
nohup /vanus/bin/gateway --config /vanus/config/gateway.yaml > /vanus/logs/gateway.log &

sleep 3600

pkill -f vanus
2 changes: 1 addition & 1 deletion build/images/controller/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ARG TARGETOS
ARG TARGETARCH
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH make build-controller

FROM alpine:3.15.4
FROM ubuntu:22.10
WORKDIR /vanus
COPY --from=builder /workspace/bin/controller bin/controller
ENTRYPOINT ["bin/controller"]
Expand Down
Loading

0 comments on commit 2bf7967

Please sign in to comment.