7551-7600 of 10000 results (55ms)
2022-01-29 §
13:53 <hashar> contint1001 and contint2001 : pruning old reflog from Zuul merger git repositories: `sudo -u zuul find /srv/zuul/git -maxdepth 4 -type d -name .git -print -execdir git reflog expire --all --expire=now \;` [production]
05:25 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcontrol2003-dev.wikimedia.org with OS bullseye [production]
04:45 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudcontrol2003-dev.wikimedia.org with OS bullseye [production]
00:14 <ebernhardson> restart elasticsearch_6@production-search-psi-eqiad on elastic1049 to address CirrusSearchJVMGCOldPoolFlatlined alert [production]
2022-01-28 §
21:52 <mutante> purging font packages from mwdebug* and scandium* T294378 [production]
21:47 <mutante> purging font packages from remaining appservers in codfw mw23* ranges.. T294378 [production]
20:14 <hnowlan@cumin1001> END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) [production]
20:10 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
20:10 <hnowlan@cumin1001> END (PASS) - Cookbook sre.postgresql.postgres-init (exit_code=0) [production]
17:52 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1024.eqiad.wmnet with OS buster [production]
17:17 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase1023.eqiad.wmnet [production]
17:17 <hnowlan@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1024.eqiad.wmnet with OS buster [production]
17:14 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1023.eqiad.wmnet with OS buster [production]
16:41 <hnowlan@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1023.eqiad.wmnet with OS buster [production]
16:41 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase1022.eqiad.wmnet [production]
16:22 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1022.eqiad.wmnet with OS buster [production]
15:50 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
15:50 <hnowlan@cumin1001> END (PASS) - Cookbook sre.postgresql.postgres-init (exit_code=0) [production]
15:47 <hnowlan@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1022.eqiad.wmnet with OS buster [production]
15:47 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase1021.eqiad.wmnet [production]
15:47 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1021.eqiad.wmnet with OS buster [production]
15:41 <vgutierrez> pool cp4031 using envoy as TLS termination layer - T271421 [production]
15:14 <Amir1> start of cleaning lint errors caused by content model changes (T298343) [production]
14:50 <hnowlan@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1021.eqiad.wmnet with OS buster [production]
14:50 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase1020.eqiad.wmnet [production]
14:47 <vgutierrez> update varnish to version 6.0.10-1wm1 on cp4036 - T300264 [production]
14:47 <Amir1> optimizing dewiki.flaggedtemplates in db2113 [production]
14:27 <vgutierrez> update varnish to version 6.0.10-1wm1 on cp4034 - T300264 [production]
13:20 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1020.eqiad.wmnet with OS buster [production]
13:01 <moritzm> installing uriparser security updates [production]
12:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 100%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19562 and previous config saved to /var/cache/conftool/dbconfig/20220128-123210-root.json [production]
12:30 <moritzm> installing libseccomp bugfix updates from bullseye point release [production]
12:28 <hnowlan@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1020.eqiad.wmnet with OS buster [production]
12:20 <vgutierrez> upload varnish 6.0.10-1wm1 to apt.wm.o (buster component/varnish6) - T300264 [production]
12:17 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 75%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19561 and previous config saved to /var/cache/conftool/dbconfig/20220128-121706-root.json [production]
12:02 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 60%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19560 and previous config saved to /var/cache/conftool/dbconfig/20220128-120201-root.json [production]
11:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 50%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19559 and previous config saved to /var/cache/conftool/dbconfig/20220128-114658-root.json [production]
11:35 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
11:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 40%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19558 and previous config saved to /var/cache/conftool/dbconfig/20220128-113154-root.json [production]
11:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 25%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19557 and previous config saved to /var/cache/conftool/dbconfig/20220128-111650-root.json [production]
11:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 20%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19556 and previous config saved to /var/cache/conftool/dbconfig/20220128-110147-root.json [production]
10:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 10%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19555 and previous config saved to /var/cache/conftool/dbconfig/20220128-104643-root.json [production]
10:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 5%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19554 and previous config saved to /var/cache/conftool/dbconfig/20220128-103140-root.json [production]
10:29 <mdipietro@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcontrol2001-dev.wikimedia.org with OS bullseye [production]
10:25 <moritzm> draining ganeti1010 for eventual reimage [production]
10:23 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on ganeti1019.eqiad.wmnet with reason: Remove from Ganeti cluster for reimage [production]
10:23 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on ganeti1019.eqiad.wmnet with reason: Remove from Ganeti cluster for reimage [production]
10:18 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1168.eqiad.wmnet with OS bullseye [production]
10:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1168 (re)pooling @ 1%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19553 and previous config saved to /var/cache/conftool/dbconfig/20220128-101636-root.json [production]
09:50 <marostegui@cumin1001> START - Cookbook sre.hosts.reimage for host db1168.eqiad.wmnet with OS bullseye [production]