1101-1150 of 10000 results (36ms)
2021-04-29 ยง
11:02 <marostegui@cumin1001> dbctl commit (dc=all): 'db1110 (re)pooling @ 50%: Repool db1110', diff saved to https://phabricator.wikimedia.org/P15656 and previous config saved to /var/cache/conftool/dbconfig/20210429-110204-root.json [production]
10:59 <moritzm> updating apt on buster (SUA 198), which eases bullseye upgrades T275873 [production]
10:56 <ladsgroup@deploy1002> Synchronized php-1.37.0-wmf.3/extensions/ContentTranslation/modules/base/mw.cx.SiteMapper.js: Backport: [[gerrit:683135|Fix CX token cookie (T281346)]] (duration: 01m 08s) [production]
10:54 <ladsgroup@deploy1002> Synchronized php-1.37.0-wmf.1/extensions/ContentTranslation/modules/base/mw.cx.SiteMapper.js: Backport: [[gerrit:683134|Fix CX token cookie (T281346)]] (duration: 01m 09s) [production]
10:47 <marostegui@cumin1001> dbctl commit (dc=all): 'db1110 (re)pooling @ 25%: Repool db1110', diff saved to https://phabricator.wikimedia.org/P15655 and previous config saved to /var/cache/conftool/dbconfig/20210429-104700-root.json [production]
10:27 <marostegui> Upgrade kernel on db1110 [production]
10:24 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1110 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15654 and previous config saved to /var/cache/conftool/dbconfig/20210429-102447-marostegui.json [production]
09:42 <volans> uploaded pynetbox 5.3.0-2 to bullseye-wikimedia on qpt.w.o [production]
09:39 <volans@deploy1002> Finished deploy [homer/deploy@e394769]: Release v0.2.8 (duration: 03m 30s) [production]
09:35 <volans@deploy1002> Started deploy [homer/deploy@e394769]: Release v0.2.8 [production]
09:01 <jynus> stop replication and checking data of db2100:s7 [production]
08:57 <marostegui> Upgrade kernel on db2133 [production]
08:51 <marostegui> Upgrade kernel on db2125 [production]
08:50 <marostegui> Upgrade kernel on db2124 [production]
08:46 <marostegui> Upgrade kernel on db2122 [production]
08:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 100%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15652 and previous config saved to /var/cache/conftool/dbconfig/20210429-084011-root.json [production]
08:39 <marostegui> Upgrade kernel on db2121 [production]
08:33 <marostegui> Upgrade kernel on db2120 [production]
08:28 <volans@deploy1002> Finished deploy [homer/deploy@89cd07c]: Release v0.2.7 (duration: 03m 08s) [production]
08:27 <marostegui> Upgrade kernel on db2115 [production]
08:25 <volans@deploy1002> Started deploy [homer/deploy@89cd07c]: Release v0.2.7 [production]
08:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 80%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15651 and previous config saved to /var/cache/conftool/dbconfig/20210429-082507-root.json [production]
08:19 <marostegui> Upgrade kernel on db2114 [production]
08:12 <marostegui> Upgrade kernel on db2109 [production]
08:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 70%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15649 and previous config saved to /var/cache/conftool/dbconfig/20210429-081004-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 60%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15648 and previous config saved to /var/cache/conftool/dbconfig/20210429-075500-root.json [production]
07:54 <marostegui> Upgrade kernel on db2089 [production]
07:48 <jynus> rolling restart of bacula hosts T273182 [production]
07:48 <marostegui@deploy1002> Synchronized wmf-config/db-eqiad.php: Repool pc1007 (duration: 01m 07s) [production]
07:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 100%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15647 and previous config saved to /var/cache/conftool/dbconfig/20210429-074625-root.json [production]
07:39 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 50%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15646 and previous config saved to /var/cache/conftool/dbconfig/20210429-073956-root.json [production]
07:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 90%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15645 and previous config saved to /var/cache/conftool/dbconfig/20210429-073122-root.json [production]
07:28 <marostegui> Stop mysql and upgrade kernel on pc1007 [production]
07:28 <marostegui@deploy1002> Synchronized wmf-config/db-eqiad.php: Depool pc1007 (duration: 01m 08s) [production]
07:24 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 40%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15644 and previous config saved to /var/cache/conftool/dbconfig/20210429-072453-root.json [production]
07:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 80%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15643 and previous config saved to /var/cache/conftool/dbconfig/20210429-071618-root.json [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 25%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15642 and previous config saved to /var/cache/conftool/dbconfig/20210429-070949-root.json [production]
07:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 75%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15641 and previous config saved to /var/cache/conftool/dbconfig/20210429-070114-root.json [production]
06:54 <marostegui@cumin1001> dbctl commit (dc=all): 'db1083 (re)pooling @ 10%: Repool db1083', diff saved to https://phabricator.wikimedia.org/P15640 and previous config saved to /var/cache/conftool/dbconfig/20210429-065445-root.json [production]
06:53 <godog> add 100G to prometheus/ops in eqiad [production]
06:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 60%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15639 and previous config saved to /var/cache/conftool/dbconfig/20210429-064611-root.json [production]
06:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 50%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15637 and previous config saved to /var/cache/conftool/dbconfig/20210429-063107-root.json [production]
06:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 40%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15636 and previous config saved to /var/cache/conftool/dbconfig/20210429-061603-root.json [production]
06:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 30%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15635 and previous config saved to /var/cache/conftool/dbconfig/20210429-060100-root.json [production]
05:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 25%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15634 and previous config saved to /var/cache/conftool/dbconfig/20210429-054556-root.json [production]
05:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 20%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15633 and previous config saved to /var/cache/conftool/dbconfig/20210429-053052-root.json [production]
05:22 <marostegui> Check tables on db1121 (this will cause lag on s4 commonswiki, on wikireplicas) [production]
05:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1121 for tables checking', diff saved to https://phabricator.wikimedia.org/P15632 and previous config saved to /var/cache/conftool/dbconfig/20210429-052146-marostegui.json [production]
05:15 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 15%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15631 and previous config saved to /var/cache/conftool/dbconfig/20210429-051549-root.json [production]
05:00 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 10%: Slowly pool into s2 db1156', diff saved to https://phabricator.wikimedia.org/P15630 and previous config saved to /var/cache/conftool/dbconfig/20210429-050045-root.json [production]