301-350 of 10000 results (36ms)
2021-05-24 §
11:06 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 1129e01745107638fee785830a1599c39379695b: Remove wgGEMentorshipMigrationStage (T279853) (duration: 00m 57s) [production]
11:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 75%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16167 and previous config saved to /var/cache/conftool/dbconfig/20210524-110508-root.json [production]
11:03 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 829c61d3dd8719546cb6f5690f75c3fad4b44aad: Deploy Growth features to newcomers on bgwiki, urwiki (T280824, T280067) (duration: 00m 56s) [production]
10:51 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
10:51 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
10:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 50%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16166 and previous config saved to /var/cache/conftool/dbconfig/20210524-105004-root.json [production]
10:35 <mbsantos@deploy1002> Finished deploy [tilerator/deploy@6bfdab5]: (no justification provided) (duration: 00m 16s) [production]
10:35 <mbsantos@deploy1002> Started deploy [tilerator/deploy@6bfdab5]: (no justification provided) [production]
10:35 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 25%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16165 and previous config saved to /var/cache/conftool/dbconfig/20210524-103501-root.json [production]
10:34 <mbsantos@deploy1002> Finished deploy [kartotherian/deploy@a9a577a]: (no justification provided) (duration: 00m 15s) [production]
10:34 <mbsantos@deploy1002> Started deploy [kartotherian/deploy@a9a577a]: (no justification provided) [production]
07:59 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 100%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16164 and previous config saved to /var/cache/conftool/dbconfig/20210524-075958-root.json [production]
07:49 <XioNoX> bump Equinix Chicago RS max prefix [production]
07:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16163 and previous config saved to /var/cache/conftool/dbconfig/20210524-074659-marostegui.json [production]
07:44 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 75%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16162 and previous config saved to /var/cache/conftool/dbconfig/20210524-074454-root.json [production]
07:29 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 50%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16161 and previous config saved to /var/cache/conftool/dbconfig/20210524-072950-root.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 25%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16160 and previous config saved to /var/cache/conftool/dbconfig/20210524-071447-root.json [production]
05:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1149 - schema change', diff saved to https://phabricator.wikimedia.org/P16159 and previous config saved to /var/cache/conftool/dbconfig/20210524-052747-marostegui.json [production]
05:13 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 100%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16158 and previous config saved to /var/cache/conftool/dbconfig/20210524-051345-root.json [production]
05:09 <legoktm> restarting mailman3 on lists1001, bounce runner crashed [production]
04:58 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 75%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16157 and previous config saved to /var/cache/conftool/dbconfig/20210524-045841-root.json [production]
04:43 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 50%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16156 and previous config saved to /var/cache/conftool/dbconfig/20210524-044337-root.json [production]
04:38 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1135.eqiad.wmnet with reason: Schema change [production]
04:38 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1135.eqiad.wmnet with reason: Schema change [production]
04:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1135', diff saved to https://phabricator.wikimedia.org/P16155 and previous config saved to /var/cache/conftool/dbconfig/20210524-043654-marostegui.json [production]
04:28 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 25%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16154 and previous config saved to /var/cache/conftool/dbconfig/20210524-042834-root.json [production]
2021-05-23 §
14:25 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: EMERGENCY: f752f8b80c57a5b7e41b91873b3eac388535ac46: enwiktionary: Raise AF emergency disable treshold+count (T283460) (duration: 00m 57s) [production]
2021-05-22 §
22:13 <legoktm> reset 2FA for User:Yuvipanda on wikitech [production]
21:07 <ryankemper> [WDQS] Pooled `wdqs1006` (caught up on lag), de-pooled `wdqs1013` (8 hours) [production]
16:35 <urbanecm> [urbanecm@mwmaint1002 ~]$ mwscript deleteEqualMessages.php cswiki --delete [production]
2021-05-21 §
22:32 <bstorm> upload nfsd-ldap: 1.2+deb10u1 to buster-wikimedia T283385 [production]
18:24 <ppchelko@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
18:22 <ppchelko@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
18:14 <ppchelko@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
17:39 <ppchelko@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
17:36 <ppchelko@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
17:29 <ppchelko@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
17:28 <legoktm@deploy1002> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 19s) [production]
17:21 <clarakosi@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
17:17 <clarakosi@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
17:09 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
17:09 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
17:07 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
17:07 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
16:40 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
16:40 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
16:16 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
16:16 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
16:14 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
16:14 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]