301-350 of 10000 results (31ms)
2021-05-11 §
05:44 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1082.eqiad.wmnet [production]
05:36 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission for hosts db1082.eqiad.wmnet [production]
05:32 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1121.eqiad.wmnet with reason: REIMAGE [production]
05:30 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1121.eqiad.wmnet with reason: REIMAGE [production]
05:11 <marostegui> Reimage db1121 to buster, this will generate lag on s4 (commonswiki) on wikireplicas T280492 [production]
05:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1121 - going to be reimaged to buster T280492', diff saved to https://phabricator.wikimedia.org/P15895 and previous config saved to /var/cache/conftool/dbconfig/20210511-051102-marostegui.json [production]
05:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1182', diff saved to https://phabricator.wikimedia.org/P15894 and previous config saved to /var/cache/conftool/dbconfig/20210511-050816-marostegui.json [production]
2021-05-10 §
23:38 <urbanecm@deploy1002> Synchronized wmf-config/CommonSettings.php: 779fb53bfd7a4d9b11f865df14f8a72adb97f33b: Update messages used for tech CoC (T280886) (duration: 00m 56s) [production]
23:32 <urbanecm@deploy1002> Synchronized wmf-config/extension-list: ba8b786c7f3a290f0747a6859fd07502eb83108f: NO-OP: Enable ChessBrowser on beta (T244075) (duration: 00m 57s) [production]
23:12 <urbanecm@deploy1002> Synchronized wmf-config/logos.php: dd6fa6504350a90c9f14c218bc972558791f0a6d: Use ptwiki 20th anniversary logos (T281925) (duration: 00m 59s) [production]
23:08 <urbanecm@deploy1002> Synchronized static/images/project-logos/: f2a76b1a6eb55749395e67d74c74a7fc5df52f1b: Add ptwiki 20th anniversary logos (T281925) (duration: 00m 58s) [production]
22:28 <eileen> civicrm revision changed from 2052d79248 to 38ac15233f, config revision is 47f21e4568 [production]
21:59 <dancy@deploy1002> Synchronized php-1.37.0-wmf.4/extensions/MediaSearch/MediaSearch.i18n.php: Backport: [[gerrit:688295|Manually include I18nUtils class (T282206)]] (duration: 00m 56s) [production]
21:45 <dancy@deploy1002> Synchronized php-1.37.0-wmf.3/extensions/MediaSearch/MediaSearch.i18n.php: Backport: [[gerrit:688294|Manually include I18nUtils class (T282206)]] (duration: 01m 01s) [production]
21:39 <legoktm> nvm, downgraded flufl.bounce on lists1001 [production]
21:26 <legoktm> upgraded flufl.bounce on lists1001 and restarted mailman3 T282348 [production]
20:44 <andrew@deploy1002> Finished deploy [horizon/deploy@2604d7b]: more deployment fixes (duration: 03m 44s) [production]
20:41 <andrew@deploy1002> Started deploy [horizon/deploy@2604d7b]: more deployment fixes [production]
20:40 <andrew@deploy1002> Finished deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 (duration: 02m 07s) [production]
20:37 <andrew@deploy1002> Started deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 [production]
20:35 <andrew@deploy1002> Finished deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 (duration: 01m 55s) [production]
20:33 <andrew@deploy1002> Started deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 [production]
20:31 <andrew@deploy1002> Finished deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 (duration: 01m 21s) [production]
20:29 <andrew@deploy1002> Started deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 [production]
20:29 <andrew@deploy1002> deploy aborted: update horizon to fix T282489 (duration: 00m 36s) [production]
20:29 <andrew@deploy1002> Started deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 [production]
20:29 <andrew@deploy1002> deploy aborted: update horizon to fix T282489 (duration: 00m 15s) [production]
20:28 <andrew@deploy1002> Started deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 [production]
20:25 <andrew@deploy1002> Finished deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 (duration: 04m 10s) [production]
20:21 <andrew@deploy1002> Started deploy [horizon/deploy@6dc83bd]: update horizon to fix T282489 [production]
18:34 <jforrester@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:632598|loginwiki: Allow users to mark Notifications as read (T264834)]] (duration: 00m 57s) [production]
18:25 <jforrester@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:677325|Disable LocalisationUpdate, part I (T158360)]] (duration: 00m 58s) [production]
18:24 <XioNoX> add cmooney to all network devices [production]
18:18 <jforrester@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:679940|[wikitech] Enable VE desktop section edit links (T280291)]] (duration: 00m 57s) [production]
18:13 <jforrester@deploy1002> Synchronized wmf-config: Config: [[gerrit:657697|wgAbuseFilterAflFilterMigrationStage: Stop setting, COMPAT_NEW is default (T269712)]] (duration: 00m 57s) [production]
18:10 <jforrester@deploy1002> Synchronized wmf-config/flaggedrevs.php: Config: [[gerrit:673306|FlaggedRevs: Stop setting wgFlaggedRevsWhitelist, now ignored]] (duration: 00m 57s) [production]
18:08 <legoktm> imported new mailman3, flufl.bounce packages to apt.wm.o [production]
16:27 <jbond42> rm -r /var/lib/routinator/repository and rebuilding repo [production]
16:23 <herron@deploy1002> Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:688281|arclamp/xenon: point all hosts to eqiad (mwlog1002) (T224565)]] (duration: 00m 59s) [production]
15:20 <elukey> restart rsyslog on rpki1001 [production]
14:32 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=maps1005.eqiad.wmnet [production]
13:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 100%: Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P15892 and previous config saved to /var/cache/conftool/dbconfig/20210510-131434-root.json [production]
12:59 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 75%: Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P15891 and previous config saved to /var/cache/conftool/dbconfig/20210510-125930-root.json [production]
12:44 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 50%: Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P15890 and previous config saved to /var/cache/conftool/dbconfig/20210510-124427-root.json [production]
12:29 <volans@cumin2002> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet with reason: Initial deploy to cumin2002 - volans@cumin2002 [production]
12:29 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 25%: Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P15889 and previous config saved to /var/cache/conftool/dbconfig/20210510-122923-root.json [production]
12:27 <volans@cumin2002> START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet with reason: Initial deploy to cumin2002 - volans@cumin2002 [production]
11:46 <Urbanecm> EU B&C window done [production]
11:41 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 3418237fdbe3eaff409bb23bf97fbba51e60337a: Disabling Education Program namespaces in Russian Wikipedia (T282112) (duration: 00m 57s) [production]
11:37 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 8bef11c3048683663e6edc38e21cd6d6d1192eb7: Add *.geograph.ie to the wgCopyUploadsDomains allowlist of Wikimedia Commons (T282007) (duration: 00m 57s) [production]