1751-1800 of 10000 results (28ms)
2021-05-24 §
13:44 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:43 <herron@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'eventstreams' for release 'production' . [production]
13:43 <herron@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . [production]
13:41 <herron@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . [production]
13:41 <herron@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . [production]
13:40 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
13:39 <herron@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . [production]
13:39 <herron@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . [production]
13:37 <herron@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
13:36 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:35 <herron@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
13:34 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
13:34 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
13:34 <herron@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
13:33 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
13:33 <herron@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
12:18 <urbanecm> Uninstalling Flow from ruwiki: Delete all pages in NS2600 (Flow's Topic) in ruwiki via deleteBatch.php (T282132; P16170) [production]
12:16 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 47e040bc6bd678e4916e0a43ad1cba5b2096274a: ruwiki: Uninstall Flow (T282132) (duration: 00m 56s) [production]
11:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1105:3311', diff saved to https://phabricator.wikimedia.org/P16169 and previous config saved to /var/cache/conftool/dbconfig/20210524-113711-marostegui.json [production]
11:20 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 100%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16168 and previous config saved to /var/cache/conftool/dbconfig/20210524-112011-root.json [production]
11:16 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1183.eqiad.wmnet with reason: Schema change [production]
11:16 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on db1183.eqiad.wmnet with reason: Schema change [production]
11:06 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 1129e01745107638fee785830a1599c39379695b: Remove wgGEMentorshipMigrationStage (T279853) (duration: 00m 57s) [production]
11:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 75%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16167 and previous config saved to /var/cache/conftool/dbconfig/20210524-110508-root.json [production]
11:03 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 829c61d3dd8719546cb6f5690f75c3fad4b44aad: Deploy Growth features to newcomers on bgwiki, urwiki (T280824, T280067) (duration: 00m 56s) [production]
10:51 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
10:51 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on maps1009.eqiad.wmnet with reason: Planet reimport [production]
10:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 50%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16166 and previous config saved to /var/cache/conftool/dbconfig/20210524-105004-root.json [production]
10:35 <mbsantos@deploy1002> Finished deploy [tilerator/deploy@6bfdab5]: (no justification provided) (duration: 00m 16s) [production]
10:35 <mbsantos@deploy1002> Started deploy [tilerator/deploy@6bfdab5]: (no justification provided) [production]
10:35 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 25%: Repool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16165 and previous config saved to /var/cache/conftool/dbconfig/20210524-103501-root.json [production]
10:34 <mbsantos@deploy1002> Finished deploy [kartotherian/deploy@a9a577a]: (no justification provided) (duration: 00m 15s) [production]
10:34 <mbsantos@deploy1002> Started deploy [kartotherian/deploy@a9a577a]: (no justification provided) [production]
07:59 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 100%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16164 and previous config saved to /var/cache/conftool/dbconfig/20210524-075958-root.json [production]
07:49 <XioNoX> bump Equinix Chicago RS max prefix [production]
07:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1099:3311', diff saved to https://phabricator.wikimedia.org/P16163 and previous config saved to /var/cache/conftool/dbconfig/20210524-074659-marostegui.json [production]
07:44 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 75%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16162 and previous config saved to /var/cache/conftool/dbconfig/20210524-074454-root.json [production]
07:29 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 50%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16161 and previous config saved to /var/cache/conftool/dbconfig/20210524-072950-root.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 25%: Repool db1135', diff saved to https://phabricator.wikimedia.org/P16160 and previous config saved to /var/cache/conftool/dbconfig/20210524-071447-root.json [production]
05:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1149 - schema change', diff saved to https://phabricator.wikimedia.org/P16159 and previous config saved to /var/cache/conftool/dbconfig/20210524-052747-marostegui.json [production]
05:13 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 100%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16158 and previous config saved to /var/cache/conftool/dbconfig/20210524-051345-root.json [production]
05:09 <legoktm> restarting mailman3 on lists1001, bounce runner crashed [production]
04:58 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 75%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16157 and previous config saved to /var/cache/conftool/dbconfig/20210524-045841-root.json [production]
04:43 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 50%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16156 and previous config saved to /var/cache/conftool/dbconfig/20210524-044337-root.json [production]
04:38 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1135.eqiad.wmnet with reason: Schema change [production]
04:38 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1135.eqiad.wmnet with reason: Schema change [production]
04:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1135', diff saved to https://phabricator.wikimedia.org/P16155 and previous config saved to /var/cache/conftool/dbconfig/20210524-043654-marostegui.json [production]
04:28 <marostegui@cumin1001> dbctl commit (dc=all): 'db1142 (re)pooling @ 25%: Repool db1142', diff saved to https://phabricator.wikimedia.org/P16154 and previous config saved to /var/cache/conftool/dbconfig/20210524-042834-root.json [production]
2021-05-23 §
14:25 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: EMERGENCY: f752f8b80c57a5b7e41b91873b3eac388535ac46: enwiktionary: Raise AF emergency disable treshold+count (T283460) (duration: 00m 57s) [production]
2021-05-22 §
22:13 <legoktm> reset 2FA for User:Yuvipanda on wikitech [production]