Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OVN for APB External Route Operations flaky #3887

Closed
tssurya opened this issue Sep 4, 2023 · 5 comments
Closed

OVN for APB External Route Operations flaky #3887

tssurya opened this issue Sep 4, 2023 · 5 comments
Assignees
@npinaeva
Labels
kind/ci-flake Flakes seen in CI

Comments

@tssurya
Copy link
Member

tssurya commented Sep 4, 2023

2023-09-04T17:52:02.3438888Z �[91m�[1m[Fail] �[0m�[90mOVN for APB External Route Operations �[0m�[0mon setting namespace gateway static hop �[0m�[90mreconciles deleting a pod with namespace double exgw static gateway already set �[0m�[91m�[1m[It] No BFD �[0m

],"QOSRules":[]}
2023-09-04T17:47:01.9785764Z I0904 17:47:01.976435   23835 cache.go:996] cache "msg"="processing update" "database"="OVN_Northbound" "table"="Logical_Switch_Port" "uuid"="19a1d13b-12a2-48bc-8798-ccbecfafd9bd"
2023-09-04T17:47:01.9787447Z I0904 17:47:01.976583   23835 cache.go:1203] cache "msg"="deleting model" "database"="OVN_Northbound" "model"={"UUID":"19a1d13b-12a2-48bc-8798-ccbecfafd9bd","Addresses":["0a:58:0a:80:01:03 10.128.1.3"],"Dhcpv4Options":null,"Dhcpv6Options":null,"DynamicAddresses":null,"Enabled":null,"ExternalIDs":{"namespace":"namespace1","pod":"true"},"HaChassisGroup":null,"MirrorRules":null,"Name":"namespace1_myPod","Options":{"iface-id-ver":"myPod","requested-chassis":"node1"},"ParentName":null,"PortSecurity":["0a:58:0a:80:01:03 10.128.1.3"],"Tag":null,"TagRequest":null,"Type":"","Up":null} "table"="Logical_Switch_Port" "uuid"="19a1d13b-12a2-48bc-8798-ccbecfafd9bd"
2023-09-04T17:47:01.9788708Z I0904 17:47:01.976672   23835 cache.go:996] cache "msg"="processing update" "database"="OVN_Northbound" "table"="Logical_Switch" "uuid"="44e8c4bd-651e-4b43-8b0c-e5d9c491b7da"
2023-09-04T17:47:01.9790493Z I0904 17:47:01.976835   23835 cache.go:1196] cache "msg"="updating model" "database"="OVN_Northbound" "new"={"UUID":"44e8c4bd-651e-4b43-8b0c-e5d9c491b7da","ACLs":null,"Copp":null,"DNSRecords":null,"ExternalIDs":null,"ForwardingGroups":null,"LoadBalancer":null,"LoadBalancerGroup":null,"Name":"node1","OtherConfig":null,"Ports":null,"QOSRules":null} "old"={"UUID":"44e8c4bd-651e-4b43-8b0c-e5d9c491b7da","ACLs":null,"Copp":null,"DNSRecords":null,"ExternalIDs":null,"ForwardingGroups":null,"LoadBalancer":null,"LoadBalancerGroup":null,"Name":"node1","OtherConfig":null,"Ports":["19a1d13b-12a2-48bc-8798-ccbecfafd9bd"],"QOSRules":null} "table"="Logical_Switch" "uuid"="44e8c4bd-651e-4b43-8b0c-e5d9c491b7da"
2023-09-04T17:47:01.9792695Z 2023/09/04 17:47:01 cache: "caller"={"file":"cache.go","line":1203} "level"=5 "msg"="deleting model" "table"="Logical_Switch_Port" "uuid"="19a1d13b-12a2-48bc-8798-ccbecfafd9bd" "model"={"UUID":"19a1d13b-12a2-48bc-8798-ccbecfafd9bd","Addresses":["0a:58:0a:80:01:03 10.128.1.3"],"Dhcpv4Options":null,"Dhcpv6Options":null,"DynamicAddresses":null,"Enabled":null,"ExternalIDs":{"pod":"true","namespace":"namespace1"},"HaChassisGroup":null,"MirrorRules":[],"Name":"namespace1_myPod","Options":{"requested-chassis":"node1","iface-id-ver":"myPod"},"ParentName":null,"PortSecurity":["0a:58:0a:80:01:03 10.128.1.3"],"Tag":null,"TagRequest":null,"Type":"","Up":null}
2023-09-04T17:47:01.9794956Z 2023/09/04 17:47:01 cache: "caller"={"file":"cache.go","line":1196} "level"=5 "msg"="updating model" "table"="Logical_Switch" "uuid"="44e8c4bd-651e-4b43-8b0c-e5d9c491b7da" "old"={"UUID":"44e8c4bd-651e-4b43-8b0c-e5d9c491b7da","ACLs":[],"Copp":null,"DNSRecords":[],"ExternalIDs":{},"ForwardingGroups":[],"LoadBalancer":[],"LoadBalancerGroup":[],"Name":"node1","OtherConfig":{},"Ports":["19a1d13b-12a2-48bc-8798-ccbecfafd9bd"],"QOSRules":[]} "new"={"UUID":"44e8c4bd-651e-4b43-8b0c-e5d9c491b7da","ACLs":[],"Copp":null,"DNSRecords":[],"ExternalIDs":{},"ForwardingGroups":[],"LoadBalancer":[],"LoadBalancerGroup":[],"Name":"node1","OtherConfig":{},"Ports":[],"QOSRules":[]}
2023-09-04T17:47:01.9796021Z I0904 17:47:01.977493   23835 pods.go:145] Attempting to release IPs for pod: namespace1/myPod, ips: 10.128.1.3
2023-09-04T17:47:01.9796493Z I0904 17:47:01.977543   23835 default_network_controller.go:671] Recording success event on pod namespace1/myPod
2023-09-04T17:47:01.9797074Z I0904 17:47:01.977710   23835 master_controller.go:541] Updating Admin Policy Based External Route policy with Status: Success, Message: Configured external gateway IPs: 9.0.0.1,9.0.0.2
2023-09-04T17:47:06.9767390Z I0904 17:47:06.976504   23835 handler.go:201] Sending *v1.Namespace event handler 1 for removal
2023-09-04T17:47:06.9768393Z I0904 17:47:06.976531   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9770640Z I0904 17:47:06.976950   23835 reflector.go:293] Stopping reflector *v1.AdminPolicyBasedExternalRoute (0s) from pkg/crd/adminpolicybasedroute/v1/apis/informers/externalversions/factory.go:116
2023-09-04T17:47:06.9772801Z I0904 17:47:06.977177   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9774601Z I0904 17:47:06.977344   23835 reflector.go:293] Stopping reflector *v1.Service (0s) from k8s.io/client-go/informers/factory.go:150
2023-09-04T17:47:06.9775574Z I0904 17:47:06.976970   23835 handler.go:215] Removed *v1.Namespace event handler 1
2023-09-04T17:47:06.9777265Z I0904 17:47:06.977380   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9778595Z I0904 17:47:06.977189   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9780555Z I0904 17:47:06.977950   23835 reflector.go:293] Stopping reflector *v1.NetworkPolicy (0s) from k8s.io/client-go/informers/factory.go:150
2023-09-04T17:47:06.9781823Z I0904 17:47:06.977821   23835 reflector.go:293] Stopping reflector *v1.Pod (0s) from k8s.io/client-go/informers/factory.go:150
2023-09-04T17:47:06.9783847Z I0904 17:47:06.978309   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9785819Z I0904 17:47:06.978474   23835 reflector.go:293] Stopping reflector *v1.Node (0s) from k8s.io/client-go/informers/factory.go:150
2023-09-04T17:47:06.9786953Z I0904 17:47:06.978640   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9788667Z I0904 17:47:06.978508   23835 handler.go:201] Sending *v1.Pod event handler 2 for removal
2023-09-04T17:47:06.9790216Z I0904 17:47:06.978787   23835 reflector.go:293] Stopping reflector *v1.Namespace (0s) from k8s.io/client-go/informers/factory.go:150
2023-09-04T17:47:06.9791265Z I0904 17:47:06.978325   23835 watch.go:183] Stopping fake watcher.
2023-09-04T17:47:06.9793300Z I0904 17:47:06.979221   23835 reflector.go:293] Stopping reflector *v1.EndpointSlice (0s) from k8s.io/client-go/informers/factory.go:150
2023-09-04T17:47:06.9794588Z I0904 17:47:06.979231   23835 handler.go:215] Removed *v1.Pod event handler 2
2023-09-04T17:47:06.9796848Z I0904 17:47:06.979590   23835 obj_retry.go:432] Stop channel got triggered: will stop retrying failed objects of type *v1.Pod
2023-09-04T17:47:06.9798392Z I0904 17:47:06.979753   23835 obj_retry.go:432] Stop channel got triggered: will stop retrying failed objects of type *v1.Namespace
2023-09-04T17:47:06.9803721Z 
2023-09-04T17:47:06.9804188Z �[91m�[1m• Failure [5.459 seconds]�[0m
2023-09-04T17:47:06.9804508Z OVN for APB External Route Operations
2023-09-04T17:47:06.9807574Z �[90m/home/runner/work/ovn-kubernetes/ovn-kubernetes/go-controller/pkg/ovn/external_gateway_apb_test.go:33�[0m
2023-09-04T17:47:06.9810384Z   on setting namespace gateway static hop
2023-09-04T17:47:06.9814161Z   �[90m/home/runner/work/ovn-kubernetes/ovn-kubernetes/go-controller/pkg/ovn/external_gateway_apb_test.go:104�[0m
2023-09-04T17:47:06.9814923Z     reconciles deleting a pod with namespace double exgw static gateway already set
2023-09-04T17:47:06.9815795Z     �[90m/home/runner/work/ovn-kubernetes/ovn-kubernetes/go-controller/pkg/ovn/external_gateway_apb_test.go:703�[0m
2023-09-04T17:47:06.9816402Z       �[91m�[1mNo BFD [It]�[0m
2023-09-04T17:47:06.9817587Z       �[90m/home/runner/work/ovn-kubernetes/ovn-kubernetes/go-controller/pkg/ovn/external_gateway_apb_test.go:768�[0m
2023-09-04T17:47:06.9820310Z 
2023-09-04T17:47:06.9820745Z       �[91mTimed out after 5.001s.
2023-09-04T17:47:06.9823426Z       Expected
2023-09-04T17:47:06.9823925Z           <[]libovsdb.TestData | len:4, cap:4>: [
2023-09-04T17:47:06.9826643Z               <*nbdb.LogicalSwitch | 0xc02a8d1340>{
2023-09-04T17:47:06.9827449Z                   UUID: "44e8c4bd-651e-4b43-8b0c-e5d9c491b7da [node1]",
2023-09-04T17:47:06.9830334Z                   ACLs: nil,
2023-09-04T17:47:06.9830787Z                   Copp: nil,
2023-09-04T17:47:06.9834607Z                   DNSRecords: nil,
2023-09-04T17:47:06.9835206Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9835725Z                   ForwardingGroups: nil,
2023-09-04T17:47:06.9836223Z                   LoadBalancer: nil,
2023-09-04T17:47:06.9836795Z                   LoadBalancerGroup: nil,
2023-09-04T17:47:06.9839474Z                   Name: "node1",
2023-09-04T17:47:06.9840031Z                   OtherConfig: nil,
2023-09-04T17:47:06.9841070Z                   Ports: nil,
2023-09-04T17:47:06.9841599Z                   QOSRules: nil,
2023-09-04T17:47:06.9842160Z               },
2023-09-04T17:47:06.9842837Z               <*nbdb.LogicalRouter | 0xc02a8f7520>{
2023-09-04T17:47:06.9843966Z                   UUID: "d5629b32-0d5b-4d32-9395-f938a1c7c8e1",
2023-09-04T17:47:06.9844758Z                   Copp: nil,
2023-09-04T17:47:06.9845585Z                   Enabled: nil,
2023-09-04T17:47:06.9846515Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9847343Z                   LoadBalancer: nil,
2023-09-04T17:47:06.9848004Z                   LoadBalancerGroup: nil,
2023-09-04T17:47:06.9848655Z                   Name: "GR_node1",
2023-09-04T17:47:06.9849161Z                   Nat: nil,
2023-09-04T17:47:06.9849610Z                   Options: nil,
2023-09-04T17:47:06.9850063Z                   Policies: nil,
2023-09-04T17:47:06.9850472Z                   Ports: nil,
2023-09-04T17:47:06.9850964Z                   StaticRoutes: [
2023-09-04T17:47:06.9851746Z                       "3b14e989-41d4-4079-a18b-cdeac1095deb",
2023-09-04T17:47:06.9852468Z                       "9f4aff21-7332-48d5-b75e-9e180dd41bd5",
2023-09-04T17:47:06.9852851Z                   ],
2023-09-04T17:47:06.9853149Z               },
2023-09-04T17:47:06.9853696Z               <*nbdb.LogicalRouterStaticRoute | 0xc02ce09960>{
2023-09-04T17:47:06.9854406Z                   UUID: "3b14e989-41d4-4079-a18b-cdeac1095deb",
2023-09-04T17:47:06.9854831Z                   BFD: nil,
2023-09-04T17:47:06.9855261Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9855747Z                   IPPrefix: "10.128.1.3/32",
2023-09-04T17:47:06.9856191Z                   Nexthop: "9.0.0.1",
2023-09-04T17:47:06.9856601Z                   Options: {
2023-09-04T17:47:06.9857170Z                       "ecmp_symmetric_reply": "true",
2023-09-04T17:47:06.9857543Z                   },
2023-09-04T17:47:06.9858105Z                   OutputPort: "rtoe-GR_node1",
2023-09-04T17:47:06.9858623Z                   Policy: "src-ip",
2023-09-04T17:47:06.9859062Z                   RouteTable: "",
2023-09-04T17:47:06.9859395Z               },
2023-09-04T17:47:06.9859945Z               <*nbdb.LogicalRouterStaticRoute | 0xc02ce09dc0>{
2023-09-04T17:47:06.9860664Z                   UUID: "9f4aff21-7332-48d5-b75e-9e180dd41bd5",
2023-09-04T17:47:06.9861085Z                   BFD: nil,
2023-09-04T17:47:06.9861501Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9861983Z                   IPPrefix: "10.128.1.3/32",
2023-09-04T17:47:06.9862434Z                   Nexthop: "9.0.0.2",
2023-09-04T17:47:06.9862826Z                   Options: {
2023-09-04T17:47:06.9863379Z                       "ecmp_symmetric_reply": "true",
2023-09-04T17:47:06.9863758Z                   },
2023-09-04T17:47:06.9864315Z                   OutputPort: "rtoe-GR_node1",
2023-09-04T17:47:06.9864822Z                   Policy: "src-ip",
2023-09-04T17:47:06.9865430Z                   RouteTable: "",
2023-09-04T17:47:06.9865826Z               },
2023-09-04T17:47:06.9866114Z           ]
2023-09-04T17:47:06.9866424Z       to consist of
2023-09-04T17:47:06.9866900Z           <[]*libovsdb.testDataMatcher | len:2, cap:2>: [
2023-09-04T17:47:06.9867269Z               {
2023-09-04T17:47:06.9867843Z                   expected: <*nbdb.LogicalSwitch | 0xc02a8d1260>{
2023-09-04T17:47:06.9868667Z                       UUID: "44e8c4bd-651e-4b43-8b0c-e5d9c491b7da [node1]",
2023-09-04T17:47:06.9869155Z                       ACLs: nil,
2023-09-04T17:47:06.9869567Z                       Copp: nil,
2023-09-04T17:47:06.9870047Z                       DNSRecords: nil,
2023-09-04T17:47:06.9870527Z                       ExternalIDs: nil,
2023-09-04T17:47:06.9871056Z                       ForwardingGroups: nil,
2023-09-04T17:47:06.9871563Z                       LoadBalancer: nil,
2023-09-04T17:47:06.9872094Z                       LoadBalancerGroup: nil,
2023-09-04T17:47:06.9872578Z                       Name: "node1",
2023-09-04T17:47:06.9873137Z                       OtherConfig: nil,
2023-09-04T17:47:06.9873583Z                       Ports: nil,
2023-09-04T17:47:06.9874158Z                       QOSRules: nil,
2023-09-04T17:47:06.9874587Z                   },
2023-09-04T17:47:06.9875010Z                   ignoreUUID: false,
2023-09-04T17:47:06.9875346Z               },
2023-09-04T17:47:06.9875641Z               {
2023-09-04T17:47:06.9876238Z                   expected: <*nbdb.LogicalRouter | 0xc02a8f7380>{
2023-09-04T17:47:06.9876880Z                       UUID: "GR_node1-UUID",
2023-09-04T17:47:06.9877316Z                       Copp: nil,
2023-09-04T17:47:06.9877745Z                       Enabled: nil,
2023-09-04T17:47:06.9878211Z                       ExternalIDs: nil,
2023-09-04T17:47:06.9878691Z                       LoadBalancer: nil,
2023-09-04T17:47:06.9879233Z                       LoadBalancerGroup: nil,
2023-09-04T17:47:06.9879730Z                       Name: "GR_node1",
2023-09-04T17:47:06.9880149Z                       Nat: nil,
2023-09-04T17:47:06.9880589Z                       Options: nil,
2023-09-04T17:47:06.9881049Z                       Policies: nil,
2023-09-04T17:47:06.9881495Z                       Ports: nil,
2023-09-04T17:47:06.9881961Z                       StaticRoutes: [],
2023-09-04T17:47:06.9882305Z                   },
2023-09-04T17:47:06.9882729Z                   ignoreUUID: false,
2023-09-04T17:47:06.9883059Z               },
2023-09-04T17:47:06.9883341Z           ]
2023-09-04T17:47:06.9883679Z       the missing elements were
2023-09-04T17:47:06.9884184Z           <[]*libovsdb.testDataMatcher | len:1, cap:1>: [
2023-09-04T17:47:06.9884554Z               {
2023-09-04T17:47:06.9885138Z                   expected: <*nbdb.LogicalRouter | 0xc02a8f7380>{
2023-09-04T17:47:06.9885751Z                       UUID: "GR_node1-UUID",
2023-09-04T17:47:06.9886189Z                       Copp: nil,
2023-09-04T17:47:06.9886630Z                       Enabled: nil,
2023-09-04T17:47:06.9887088Z                       ExternalIDs: nil,
2023-09-04T17:47:06.9887570Z                       LoadBalancer: nil,
2023-09-04T17:47:06.9888075Z                       LoadBalancerGroup: nil,
2023-09-04T17:47:06.9888651Z                       Name: "GR_node1",
2023-09-04T17:47:06.9889057Z                       Nat: nil,
2023-09-04T17:47:06.9889481Z                       Options: nil,
2023-09-04T17:47:06.9889932Z                       Policies: nil,
2023-09-04T17:47:06.9890365Z                       Ports: nil,
2023-09-04T17:47:06.9890833Z                       StaticRoutes: [],
2023-09-04T17:47:06.9891184Z                   },
2023-09-04T17:47:06.9891608Z                   ignoreUUID: false,
2023-09-04T17:47:06.9891931Z               },
2023-09-04T17:47:06.9892213Z           ]
2023-09-04T17:47:06.9892544Z       the extra elements were
2023-09-04T17:47:06.9892974Z           <[]interface {} | len:3, cap:4>: [
2023-09-04T17:47:06.9893496Z               <*nbdb.LogicalRouter | 0xc02a8f7520>{
2023-09-04T17:47:06.9894363Z                   UUID: "d5629b32-0d5b-4d32-9395-f938a1c7c8e1",
2023-09-04T17:47:06.9894883Z                   Copp: nil,
2023-09-04T17:47:06.9895285Z                   Enabled: nil,
2023-09-04T17:47:06.9895712Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9896147Z                   LoadBalancer: nil,
2023-09-04T17:47:06.9896613Z                   LoadBalancerGroup: nil,
2023-09-04T17:47:06.9897050Z                   Name: "GR_node1",
2023-09-04T17:47:06.9897436Z                   Nat: nil,
2023-09-04T17:47:06.9897832Z                   Options: nil,
2023-09-04T17:47:06.9898247Z                   Policies: nil,
2023-09-04T17:47:06.9898631Z                   Ports: nil,
2023-09-04T17:47:06.9899047Z                   StaticRoutes: [
2023-09-04T17:47:06.9899750Z                       "3b14e989-41d4-4079-a18b-cdeac1095deb",
2023-09-04T17:47:06.9900430Z                       "9f4aff21-7332-48d5-b75e-9e180dd41bd5",
2023-09-04T17:47:06.9900811Z                   ],
2023-09-04T17:47:06.9901118Z               },
2023-09-04T17:47:06.9901719Z               <*nbdb.LogicalRouterStaticRoute | 0xc02ce09960>{
2023-09-04T17:47:06.9902451Z                   UUID: "3b14e989-41d4-4079-a18b-cdeac1095deb",
2023-09-04T17:47:06.9902872Z                   BFD: nil,
2023-09-04T17:47:06.9903292Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9903763Z                   IPPrefix: "10.128.1.3/32",
2023-09-04T17:47:06.9904215Z                   Nexthop: "9.0.0.1",
2023-09-04T17:47:06.9904601Z                   Options: {
2023-09-04T17:47:06.9905154Z                       "ecmp_symmetric_reply": "true",
2023-09-04T17:47:06.9905519Z                   },
2023-09-04T17:47:06.9906076Z                   OutputPort: "rtoe-GR_node1",
2023-09-04T17:47:06.9906589Z                   Policy: "src-ip",
2023-09-04T17:47:06.9907012Z                   RouteTable: "",
2023-09-04T17:47:06.9907334Z               },
2023-09-04T17:47:06.9907873Z               <*nbdb.LogicalRouterStaticRoute | 0xc02ce09dc0>{
2023-09-04T17:47:06.9908580Z                   UUID: "9f4aff21-7332-48d5-b75e-9e180dd41bd5",
2023-09-04T17:47:06.9909000Z                   BFD: nil,
2023-09-04T17:47:06.9909416Z                   ExternalIDs: nil,
2023-09-04T17:47:06.9909900Z                   IPPrefix: "10.128.1.3/32",
2023-09-04T17:47:06.9910339Z                   Nexthop: "9.0.0.2",
2023-09-04T17:47:06.9910740Z                   Options: {
2023-09-04T17:47:06.9911285Z                       "ecmp_symmetric_reply": "true",
2023-09-04T17:47:06.9911648Z                   },
2023-09-04T17:47:06.9912178Z                   OutputPort: "rtoe-GR_node1",
2023-09-04T17:47:06.9912690Z                   Policy: "src-ip",
2023-09-04T17:47:06.9913120Z                   RouteTable: "",
2023-09-04T17:47:06.9913441Z               },
2023-09-04T17:47:06.9913772Z           ]�[0m
2023-09-04T17:47:06.9914216Z 
2023-09-04T17:47:06.9914841Z       /home/runner/work/ovn-kubernetes/ovn-kubernetes/go-controller/pkg/ovn/external_gateway_apb_test.go:761
2023-09-04T17:47:06.9915334Z �[90m------------------------------�[0m
2023-09-04T17:47:06.9916071Z �[0mOVN for APB External Route Operations�[0m �[90mon setting namespace gateway static hop�[0m �[0mreconciles deleting a pod with namespace double exgw static gateway already set�[0m 
2023-09-04T17:47:06.9916665Z   �[1mBFD�[0m
2023-09-04T17:47:06.9917268Z   �[37m/home/runner/work/ovn-kubernetes/ovn-kubernetes/go-controller/pkg/ovn/external_gateway_apb_test.go:838�[0m
2023-09-04T17:47:07.0114458Z 2023/09/04 17:47:07 cache: "caller"={"file":"cache.go","line":1189} "level"=5 "msg"="inserting model" "table"="Database" "uuid"="93275282-baa4-4adb-a8f8-0cf51f3966a0" "model"={"UUID":"93275282-baa4-4adb-a8f8-0cf51f3966a0","Cid":null,"Connected":true,"Index":null,"Leader":true,"Model":"clustered","Name":"OVN_Northbound","Schema":null,"Sid":"ececa0ba"}
2023-09-04T17:47:07.0116541Z 2023/09/04 17:47:07 cache: "caller"={"file":"cache.go","line":1189} "level"=5 "msg"="inserting model" "table"="Database" "uuid"="93275282-baa4-4adb-a8f8-0cf51f3966a0" "model"={"UUID":"93275282-baa4-4adb-a8f8-0cf51f3966a0","Cid":null,"Connected":true,"Index":null,"Leader":true,"Model":"clustered","Name":"OVN_Northbound","Schema":null,"Sid":"ececa0ba"}
@tssurya
Copy link
Member Author

tssurya commented Sep 4, 2023

https://github.com/ovn-org/ovn-kubernetes/actions/runs/6076284846/job/16484014897?pr=3886

@npinaeva npinaeva self-assigned this Oct 4, 2023
@jcaamano
Copy link
Contributor

jcaamano commented Oct 4, 2023

There are several APB flakey tests and they might have a common cause. I don''t want to bug you down @npinaeva but I think it makes sense to have one issue for all of them initially. We can split them out eventually and ask for additional help. In the meantime I will just add them here:

OVN External Gateway policy when updating a policy validates that changing a static hop from an existing policy will be applied to the target namespaces
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6400303742/job/17373732661
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6354239470/job/17260380226

OVN External Gateway policy when updating a policy validates that removing a duplicated static hop IP from an overlapping policy static hop will keep the static IP in the route policy
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6392507618/job/17349903195
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6383859850/job/17325329155
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6383512722/job/17324272379
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6355511855/job/17263729496
https://github.com/ovn-org/ovn-kubernetes/actions/runs/6351611320/job/17253091707
...

@npinaeva
Copy link
Member

npinaeva commented Oct 4, 2023

these "when updating a policy" tests were discussed here #3896, and I know the root cause, and they should be fixed with the status manager.
I think it makes sense to have one issue to help people find what we already know (at least till it becomes bigger that the repo itself)
I am also testing the failure @tssurya reported, and I will give updates for different test cases here

@npinaeva npinaeva mentioned this issue Oct 4, 2023
Closed
@martinkennelly martinkennelly mentioned this issue Oct 5, 2023
Merged
@martinkennelly
Copy link
Member

https://github.com/ovn-org/ovn-kubernetes/actions/runs/6355511855/job/17263729496?pr=3921

2023-09-29T18:44:50.9589809Z �[91m�[1m[Fail] �[0m�[90mOVN External Gateway policy �[0m�[0mwhen updating a policy �[0m�[91m�[1m[It] validates that removing a duplicated static hop IP from an overlapping policy static hop will keep the static IP in the route policy �[0m

@trozet trozet added the kind/ci-flake Flakes seen in CI label Oct 12, 2023
@npinaeva
Copy link
Member

npinaeva commented Dec 4, 2023

should be fixed by #3750

@npinaeva npinaeva closed this as completed Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@npinaeva npinaeva

Labels
kind/ci-flake Flakes seen in CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tssurya @trozet @npinaeva @martinkennelly @jcaamano