151-200 of 10000 results (36ms)
2020-08-04 ยง
13:56 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:56 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . [production]
13:56 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'kube-system' for release 'rbac-deploy-clusterrole' . [production]
13:51 <hashar> Install newer openjdk on contint2001 and restarting CI Jenkins [production]
12:00 <jayme> helm was updated: 2.16.7-2 -> 2.16.9-1 on chartmuseum*, contint*, deploy* [production]
11:43 <Lucas_WMDE> EU backport window done [production]
11:41 <marostegui> Deploy schema change on s3 codfw master, lag might show up on codfw s3 T259238 [production]
11:37 <moritzm> installing openjdk-11 security updates [production]
11:36 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:618266|Load WikibaseRepo using extension registration in production (T257433)]] (duration: 00m 58s) [production]
11:12 <Lucas_WMDE> Deployed patch for T86738 / T259565 [production]
11:03 <moritzm> installing e2fsprogs security updates for stretch [production]
10:47 <moritzm> installing tomcat8 security updates [production]
10:47 <vgutierrez> upgrade acme-chief to version 0.28 [production]
10:32 <vgutierrez> upload acme-chief 0.28 to apt.wm.o (buster) - T259338 [production]
10:18 <moritzm> installing imagemagick security updates on stretch [production]
10:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1101:3317 for MCR and PK change T259524', diff saved to https://phabricator.wikimedia.org/P12156 and previous config saved to /var/cache/conftool/dbconfig/20200804-100035-marostegui.json [production]
09:56 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1098:3317', diff saved to https://phabricator.wikimedia.org/P12155 and previous config saved to /var/cache/conftool/dbconfig/20200804-095608-marostegui.json [production]
09:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1098:3317', diff saved to https://phabricator.wikimedia.org/P12154 and previous config saved to /var/cache/conftool/dbconfig/20200804-094909-marostegui.json [production]
09:06 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:04 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:58 <moritzm> installing python3.5 security updates [production]
08:15 <moritzm> installing remaining cups security updates [production]
08:13 <XioNoX> cleaning up a bunch of prefix limit reached issues [production]
08:00 <marostegui> Failover m2 from db1132 to db1107 -T257540 [production]
07:54 <moritzm> installing poppler security updates on stretch [production]
07:43 <jayme> imported helm_2.16.9-1 to jessie-wikimedia [production]
07:43 <jayme> imported helm_2.16.9-1 to stretch-wikimedia [production]
07:38 <jayme> imported helm_2.16.9-1 to buster-wikimedia [production]
07:34 <elukey> upgrade druid analytics (backend for Turnilo/Superset/etc..) to 0.19 [production]
07:32 <XioNoX> remove nonstop-bridging from fasw-c-eqiad switches - T191667 [production]
07:29 <XioNoX> remove nonstop-bridging from eqiad asw2 switches - T191667 [production]
07:28 <XioNoX> remove nonstop-bridging from asw2-esams - T191667 [production]
07:27 <marostegui> Start topology changes on m2 - T257540 [production]
07:25 <moritzm> installing rails security updates [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1119', diff saved to https://phabricator.wikimedia.org/P12153 and previous config saved to /var/cache/conftool/dbconfig/20200804-064223-marostegui.json [production]
06:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1119', diff saved to https://phabricator.wikimedia.org/P12152 and previous config saved to /var/cache/conftool/dbconfig/20200804-063026-marostegui.json [production]
06:27 <_joe_> restarting docker daemon on kubestage1002, seems like a case of https://github.com/moby/moby/issues/29635 [production]
06:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Restore original weight to db1089 on main traffic', diff saved to https://phabricator.wikimedia.org/P12151 and previous config saved to /var/cache/conftool/dbconfig/20200804-062358-marostegui.json [production]
06:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1119', diff saved to https://phabricator.wikimedia.org/P12150 and previous config saved to /var/cache/conftool/dbconfig/20200804-062256-marostegui.json [production]
06:19 <oblivian@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . [production]
06:13 <tstarling@deploy1001> Synchronized wmf-config/CommonSettings.php: re-enabling lilypond execution in safe mode 3rd attempt (duration: 00m 58s) [production]
06:12 <marostegui@cumin1001> dbctl commit (dc=all): 'More weight to db1089 on main traffic', diff saved to https://phabricator.wikimedia.org/P12149 and previous config saved to /var/cache/conftool/dbconfig/20200804-061255-marostegui.json [production]
06:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1119', diff saved to https://phabricator.wikimedia.org/P12148 and previous config saved to /var/cache/conftool/dbconfig/20200804-061209-marostegui.json [production]
06:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1098:3317 for MCR', diff saved to https://phabricator.wikimedia.org/P12147 and previous config saved to /var/cache/conftool/dbconfig/20200804-061003-marostegui.json [production]
05:37 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
05:35 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
05:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1119 for reimage', diff saved to https://phabricator.wikimedia.org/P12146 and previous config saved to /var/cache/conftool/dbconfig/20200804-051843-marostegui.json [production]
05:04 <marostegui> Reboot db1107 to pick up the last kernel [production]
05:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1089 into API', diff saved to https://phabricator.wikimedia.org/P12145 and previous config saved to /var/cache/conftool/dbconfig/20200804-050150-marostegui.json [production]
03:56 <legoktm> added Arlo to wmf-deployment Gerrit group [production]