New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature: k0s init, import some dependencies form k0sproject. #1619

Merged

allencloud merged 3 commits into sealerio:main from starComingup:feat/k0s-init

Aug 14, 2022

Member

starComingup commented Aug 5, 2022 •

edited

Loading

Signed-off-by: starComingup [email protected]

Describe what this PR does / why we need it

add k0s init step feature for sealer, it prepare k0sctl install env and image registry and so on.

Does this pull request fix one issue?

NONE, but is a step for k0s clusterRuntime.

Describe how you did it

NONE

Describe how to verify it

use unit test for check it work!

Todo list:

k.MergeConfigOnMaster0, // convert cluster file to k0s config
k.SSHCopyIDToHosts, // ssh-key-gen and ssh-copy-id to login hosts without passwd
k.GenerateCert, // generate Registry cert mainly
k.ApplyRegistryOnMaster0, // start a private registry for cluster

Note:
Why we don't need initMaster0 and GetKubectlConfig func like k8s init step?
In k0sctl design which lead the cluster creation, cluster init phase and node join phase integrate to k0sctl apply and it will reconcile by k0sctl itself, so to keep sealer now createProcess logic c.init and c.join. I would like to implement the k0sctl apply cmd in join function rather than init function. And after apply, and we can use k0sctl kubeconfig to create the config file for kubectl.

starComingup requested a review from kakaZhou719

August 5, 2022 09:22

github-actions bot added the test label

Member Author

starComingup commented Aug 5, 2022

I use go mod tidy, but it seems so many dependencies imported.

starComingup force-pushed the feat/k0s-init branch from 1f04509 to 91911e3 Compare

August 5, 2022 09:35

starComingup added the feature label

starComingup force-pushed the feat/k0s-init branch from 91911e3 to 20d38d5 Compare

August 8, 2022 10:00

starComingup changed the title ~~[WIP] feature: k0s init, import some dependencies form k0sproject.~~ feature: k0s init, import some dependencies form k0sproject.

starComingup force-pushed the feat/k0s-init branch from 20d38d5 to 53e1ed3 Compare

August 9, 2022 02:13

kakaZhou719 reviewed

View reviewed changes

pkg/runtime/k0s/init.go

+              func (k *Runtime) init() error {
+              	pipeline := []func() error{
+              		k.MergeConfigOnMaster0,
+              		k.SSHKeyGenAndCopyIDToHosts,

Member

kakaZhou719 Aug 9, 2022 •

edited

Loading

I'm not very familiar with k0s, I have a question, is this necessary to configure ssh key login？

Member Author

starComingup Aug 9, 2022

if we don't ssh-keygen id_rsa config: when k0sctl lead to install cluster, it will report:ERRO [ssh] 172.16.161.63:22: attempt 1 of 60.. failed to connect: stat /root/.ssh/id_rsa: no such file or directory
if we don't ssh-copy-id: k0sctl will report :failed to connect: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

pkg/runtime/k0s/init.go

+              	pipeline := []func() error{
+              		k.MergeConfigOnMaster0,
+              		k.SSHKeyGenAndCopyIDToHosts,
+              		k.GenerateCert,

Member

kakaZhou719 Aug 9, 2022

This part of configuring private image repositories will be removed from the runtime pkg after we refactor run process, and i think k8s runtime also need those logic. see PR #1621

Member Author

starComingup Aug 9, 2022

Got it! will adapt it when new code merged.

Member

allencloud Aug 11, 2022

I suggest that we do this before the code merging. Or at least you add a TODO comment

// TODO: move all these registry operation to the specific registry packages.

pkg/runtime/k0s/k0sctl/v1beta1/types.go Outdated

+              //
+              //     http://www.apache.org/licenses/LICENSE-2.0
+              //
+              // Unless required by applicable law or agreed to in writing, software

Member

kakaZhou719 Aug 9, 2022 •

edited

Loading

If this code is from the k0s project, please keep the k0s license. Or could we use those struct directly ?

Member Author

starComingup Aug 10, 2022

Let me keep original license.

pkg/runtime/k0s/k0sctl/k0sctl_config.go

+              	return c.convertIPVSToAddress(clusterFile)
+              }
+              func (c *K0sConfig) convertIPVSToAddress(clusterFile *v2.Cluster) error {

Member

kakaZhou719 Aug 9, 2022

I think the logic here seems to be parsing the host configuration information in the *v2.Cluster structure. Do we need to rename this function? convertClusterHosts ?

Member Author

starComingup Aug 10, 2022

This func mainly use sealer cluster file spec Hosts:IPS as input and out put k0s cluster spec Hosts:SSH:Address, so i named this. If there is a better way to rename it? I think though convertClusterHosts seems like convert some thing in Hosts field, but it don't represent we convert Hosts every field under Hosts.

pkg/runtime/k0s/k0sctl/k0sctl_config.go Outdated

+              		return err
+              	}
+              	if err := osi.NewAtomicWriter(rootfs + "/k0sctl.yaml").WriteFile(marshal); err != nil {
+              		return fmt.Errorf("write k0sctl config error: %s", err)

Member

kakaZhou719 Aug 9, 2022

failed to write k0sctl config: %v ,err

pkg/runtime/k0s/k0sctl/k0sctl_config.go Outdated

+              			}
+              		}
+              	}
+              	return 0, fmt.Errorf("ssh port parse failed")

Member

kakaZhou719 Aug 9, 2022

failed to parse ssh port

pkg/runtime/k0s/k0sctl/k0sctl_config.go Outdated

+              }
+              func (c *K0sConfig) joinK0sHosts(ipList []string, hosts []v2.Host, ssh v1.SSH, role string) error {
+              	for _, masterIP := range ipList {

Member

kakaZhou719 Aug 9, 2022

masterIP-->> ip

pkg/runtime/k0s/k0sctl/k0sctl_config.go

+              // parseSSHPortByIP parse cluster file ssh port to k0s ssh port.
+              func (c *K0sConfig) parseSSHPortByIP(ipAddr string, hosts []v2.Host, ssh v1.SSH) (int, error) {
+              	for _, host := range hosts {

Member

kakaZhou719 Aug 9, 2022

The logic here should be able to simplify。
1.convert hosts to map[string]v1.SSH with ssh arg.
2. get map value from ipAddr.

Member

allencloud commented Aug 10, 2022

Good job, 👍 I think @sealerio/sealer-maintainers could review this first, and @starComingup can illustrate this pull request in the maintainer's meeting this night.

allencloud reviewed

View reviewed changes

pkg/runtime/k0s/k0sctl/k0sctl_config.go Outdated

+              // addHostField generate k0s config spec Host and append it to config file.
+              func (c *K0sConfig) addHostField(ipAddr string, port int, role, user string) {
+              	c2 := c.Spec.Hosts

Member

allencloud Aug 11, 2022

Why not just

host := cluster.Host{
      xxx
}
c.Spec.Hosts = append(c.Spec.Hosts, host)

? @starComingup

allencloud reviewed

View reviewed changes

pkg/runtime/k0s/k0sctl/v1beta1/types.go

+              }
+              // Cluster describes launchpad.yaml configuration
+              type Cluster struct {

Member

allencloud Aug 11, 2022

I am afraid that we do not think this struct definition. Just make

type K0sConfig struct{
         APIVersion string           `yaml:"apiVersion"`
	Kind       string           `yaml:"kind"`
	Metadata   *ClusterMetadata `yaml:"metadata"`
	Spec       *cluster.Spec    `yaml:"spec"`
}

?

Member Author

starComingup Aug 11, 2022

This struct is copy from k0s project. if we need to change it?

allencloud reviewed

View reviewed changes

pkg/runtime/k0s/runtime.go Outdated

+              	*sync.Mutex
+              	// cluster is sealer clusterFile
+              	cluster *v2.Cluster
+              	*Config

Member

allencloud Aug 11, 2022 •

edited

Loading

I think there is no need to set this anonymous field.

just

// Runtime struct is the runtime interface for k0s
type Runtime struct {
	*sync.Mutex
	// cluster is sealer clusterFile
	cluster *v2.Cluster
        Vlog      int
	RegConfig *registry.Config
	// 	clusterFileK0sConfig is k0sctl cluster config file
	K0sConfig *k0sctl.K0sConfig
}

allencloud reviewed

View reviewed changes

pkg/runtime/k0s/runtime.go Outdated

+              // Runtime struct is the runtime interface for k0s
+              type Runtime struct {
+              	*sync.Mutex

Member

allencloud Aug 11, 2022

I do not see any place where used this Mutex. I am wondering if this is redundant? @starComingup

Member Author

starComingup Aug 11, 2022

Copy form k8s, now remove it!

allencloud reviewed

View reviewed changes

pkg/runtime/k0s/runtime.go Outdated

+              		return fmt.Errorf("hosts spec cannot be empty")
+              	}
+              	if k.cluster.GetMaster0IP() == nil {
+              		return fmt.Errorf("controller hosts ip cannot be empty")

Member

allencloud Aug 11, 2022

the error message seems to be not so readable. Just usemaster0 IP cannot be empty. Since no one could recognize what does controller mean.

Member

allencloud commented Aug 11, 2022

Good job for the k0s initial work. 👍

I think there is still more work to do, maybe in the follow-up pull requests:

k0s testing;
other operations implementation about k0s.

LGTM unless all review comments has been finished. and wish to see that we could iterate quickly with small steps. @starComingup

justadogistaken reviewed

View reviewed changes

pkg/runtime/k0s/init.go


		k.K0sConfig.DefineConfigFork0s(string(output), k.RegConfig.Domain, k.RegConfig.Port, k.cluster.Name)

		//write k0sctl.yaml to master0 rootfs

Member

justadogistaken Aug 11, 2022

I think the name of this func explicitly illustrates its ability. Maybe we don't have to annotate it. (It's ok to keep it.)

pkg/runtime/k0s/k0sctl/v1beta1/types.go

+              	}
+              	c.Spec = &cluster.Spec{}
+              	type clusterConfig Cluster

Member

justadogistaken Aug 11, 2022

This is weird

pkg/runtime/k0s/runtime.go

+              		return nil, err
+              	}
+              	if logrus.GetLevel() == logrus.DebugLevel {

Member

justadogistaken Aug 11, 2022

How about create a function for level transform?

Member Author

starComingup Aug 11, 2022

This would be fine, more readable.

pkg/runtime/k0s/registry.go

+              const (
+              	SeaHub                      = "sea.hub"
+              	RemoteAddEtcHosts           = "cat /etc/hosts |grep '%s' || echo '%s' >> /etc/hosts"
+              	ContainerdLoginCommand      = "nerdctl login -u %s -p %s %s"

Member

justadogistaken Aug 11, 2022

Is this a general command? I think its readability fails to be better than writing in the code.

Member Author

starComingup Aug 11, 2022

This will use by join phase, so i define it as a const.

starComingup force-pushed the feat/k0s-init branch from 53e1ed3 to 8ad8187 Compare

August 11, 2022 09:47

starComingup added 2 commits

August 11, 2022 18:02


          add feature k0s init, import some dependencies form k0sproject.

0fd655e

Signed-off-by: starComingup <[email protected]>


          add README about k0s runtime module

Signed-off-by: starComingup <[email protected]>

starComingup force-pushed the feat/k0s-init branch from 167781d to 7681037 Compare

August 11, 2022 10:06


          modify k0s cluster cert.

b5b47d8

Signed-off-by: starComingup <[email protected]>

Member

allencloud commented Aug 12, 2022

What is more, I think we could add a design document about different cluster runtime support into the sealer docs directory, which could also be a follow-up pull request. @starComingup

Member Author

starComingup commented Aug 12, 2022

What is more, I think we could add a design document about different cluster runtime support into the sealer docs directory, which could also be a follow-up pull request. @starComingup

OK, I will keep this in mind and after k0s、k3s all finished and stable, I would like to design it with use cases.

allencloud approved these changes

View reviewed changes

Member

allencloud commented Aug 14, 2022

LGTM, and this is related to sealerio/basefs#23 . Please help complete the k3s other part. @starComingup

allencloud merged commit d2ad51f into sealerio:main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels