51-100 of 10000 results (63ms)
2023-05-02 ยง
16:11 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: service=thumbor,name=kubernetes101[0123].eqiad.wmnet [production]
16:10 <hnowlan@deploy1002> helmfile [eqiad] DONE helmfile.d/services/thumbor: sync [production]
16:08 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/services/thumbor: sync [production]
16:06 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: service=thumbor,name=kubernetes101[0123].eqiad.wmnet [production]
15:39 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:38 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
15:36 <claime> Re-running puppet on failed parse servers - T313227 [production]
15:35 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12 days, 0:00:00 on wdqs2022.codfw.wmnet with reason: attempting WDQS stack on bullseye [production]
15:35 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:34 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 12 days, 0:00:00 on wdqs2022.codfw.wmnet with reason: attempting WDQS stack on bullseye [production]
15:33 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
15:16 <jiji@cumin1001> END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) pool all active/active services in codfw: codfw row C switches upgrade - T334049 [production]
15:13 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:12 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
15:09 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/services/thumbor: apply [production]
15:04 <claime> enabling puppet on parse2014 [production]
15:04 <claime> enabling puppet on parse2013 [production]
15:02 <akosiaris> enable puppet on parse1005 [production]
15:00 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:00 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/services/thumbor: apply [production]
14:59 <jiji@cumin1001> START - Cookbook sre.discovery.datacenter pool all active/active services in codfw: codfw row C switches upgrade - T334049 [production]
14:59 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
14:58 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
14:56 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:55 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:54 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
14:53 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: apply [production]
14:52 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: apply [production]
14:40 <moritzm> installing intel-microcode security updates on bullseye servers [production]
14:40 <akosiaris> emergency disabling of puppet on parse hosts [production]
14:33 <akosiaris@deploy1002> helmfile [staging] DONE helmfile.d/services/machinetranslation: sync [production]
14:33 <claime> Merging new internal certs for api, jobrunner, appservers, parsoid - T313227 [production]
14:29 <akosiaris@deploy1002> helmfile [staging] START helmfile.d/services/machinetranslation: sync [production]
14:27 <denisse> sync prometheus3001 -> prometheus3002 [production]
14:27 <akosiaris@deploy1002> helmfile [staging] DONE helmfile.d/services/machinetranslation: apply [production]
14:23 <_joe_> also on contint1002, the current ci master [production]
14:22 <_joe_> restarted zuul on contint2001 [production]
14:07 <akosiaris@deploy1002> helmfile [staging] START helmfile.d/services/machinetranslation: apply [production]
13:51 <sukhe> run authdns-update to repool codfw [production]
13:47 <cgoubert@cumin1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=parsoid [production]
13:47 <cgoubert@cumin1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=appserver [production]
13:47 <cgoubert@cumin1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=api_appserver [production]
13:45 <akosiaris@deploy1002> helmfile [staging] DONE helmfile.d/services/machinetranslation: apply [production]
13:37 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:914267|Enable Kartographer Nearby on mobile (T333137)]], [[gerrit:914286|Fix clearing wrong container when closing fullscreen map (T335648)]], [[gerrit:914287|Fix clearing wrong container when closing fullscreen map (T335648)]] (duration: 14m 54s) [production]
13:25 <akosiaris@deploy1002> helmfile [staging] START helmfile.d/services/machinetranslation: apply [production]
13:24 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: name=ldap-replica2005.wikimedia.org [production]
13:24 <urbanecm@deploy1002> wmde-fisch and urbanecm: Backport for [[gerrit:914267|Enable Kartographer Nearby on mobile (T333137)]], [[gerrit:914286|Fix clearing wrong container when closing fullscreen map (T335648)]], [[gerrit:914287|Fix clearing wrong container when closing fullscreen map (T335648)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
13:22 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:914267|Enable Kartographer Nearby on mobile (T333137)]], [[gerrit:914286|Fix clearing wrong container when closing fullscreen map (T335648)]], [[gerrit:914287|Fix clearing wrong container when closing fullscreen map (T335648)]] [production]
13:16 <urbanecm@deploy1002> Sync cancelled. [production]
13:16 <urbanecm@deploy1002> urbanecm and wmde-fisch: Backport for [[gerrit:914267|Enable Kartographer Nearby on mobile (T333137)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]