1-50 of 10000 results (21ms)
2020-11-10 §
22:14 <mholloway-shell@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . [production]
22:14 <mholloway-shell@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
22:08 <mholloway-shell@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . [production]
22:08 <mholloway-shell@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
22:05 <mholloway-shell@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . [production]
21:59 <mholloway-shell@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
21:58 <jgleeson> update civicrm revision changed from c36a5cc1b1 to d0cd7f6dbb [production]
21:57 <mholloway-shell@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
21:55 <mholloway-shell@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . [production]
21:47 <ebernhardson> unban elastic1050 from eqiad search psi cluster [production]
21:28 <cstone> civicrm revision changed from b1342c4129 to c36a5cc1b1 [production]
21:24 <brennen@deploy1001> sync-file aborted: Testing: README.md sync-file with ssh -n for T223287 (duration: 00m 37s) [production]
21:23 <brennen> testing some scap operations, modified to use ssh -n for debugging T223287 [production]
21:11 <ebernhardson> ban elastic1050 from eqiad psi cluster due to excessive load [production]
21:02 <brennen@deploy1001> Finished scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] (duration: 34m 46s) [production]
20:27 <brennen@deploy1001> Started scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] [production]
20:10 <brennen@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:640254|Turn on formatnum logging (T267587, T267370)]] (duration: 01m 02s) [production]
19:06 <hknust> holger mwmaint1002 Stop T219279 [production]
18:31 <hknust> holger mwmaint1002 Start T219279 [production]
17:57 <effie> pool mw1263 mw1264 [production]
17:31 <effie> briefly depool mw1263 and mw1264 [production]
17:30 <jynus> about to shutdown db1139 for hw maintenance T261405 [production]
17:13 <dwisehaupt> upping thank you mail flow through frmx's to 30% of the total runs [production]
16:32 <XioNoX> add cloud-storage1-b-codfw to, well, codfw switches - T267378 [production]
16:20 <effie> pool mw1263 [production]
16:17 <hashar> Restarting Gerrit on gerrit1001 [production]
16:12 <hashar> Restarted Gerrit on gerrit2001 for config change [production]
15:53 <zpapierski@deploy1001> Finished deploy [wikimedia/discovery/analytics@1ab89ed]: Deploying venv workaround for Debian 9 (duration: 01m 06s) [production]
15:52 <zpapierski@deploy1001> Started deploy [wikimedia/discovery/analytics@1ab89ed]: Deploying venv workaround for Debian 9 [production]
15:38 <moritzm> installing 4.19.152 kernel packages on buster hosts (only installing the package, reboots will happen separately) [production]
15:28 <effie> depool mw1263 - T244340 [production]
15:09 <ejegg> updated fundraising python tools from 087a596d3a to 7853f426ee [production]
14:21 <effie> pooling mw1276 - T244340 [production]
13:51 <moritzm> imported php-memcached 3.0.1+2.2.0-1~wmf3+buster1 to component/php72 for buster-wikimedia [production]
13:28 <marostegui> Restart db2093 to pick up report_host - T266483 [production]
13:17 <marostegui> Restart db1117* to pick up report_host - T266483 [production]
12:46 <effie> depool mw1276 to install onhost memcached - T244340 [production]
12:33 <Lucas_WMDE> EU backport&config window done [production]
12:33 <moritzm> installing wireshark security updates [production]
12:31 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:636095|Switch parser cache to using "mcrouter-with-onhost-tier" (T264604)]] (duration: 00m 57s) [production]
12:23 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/mc.php: Config: [[gerrit:636094|Add "mcrouter-with-onhost-tier" entry to $wgObjectCaches (T264604)]] (duration: 00m 57s) [production]
12:04 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.36.0-wmf.16/extensions/Wikibase: Backport: [[gerrit:639035|Revert JS parser commits (T266671)]] (duration: 01m 04s) [production]
08:59 <hashar> Restarted Gerrit for plugins deployment [production]
08:06 <hashar> Restarting Gerrit on gerrit2001 / gerrit-replica [production]
08:04 <hashar@deploy1001> Finished deploy [gerrit/gerrit@5a41181]: jmx and prometheus metrics reporters - T184086 (duration: 00m 10s) [production]
08:04 <hashar@deploy1001> Started deploy [gerrit/gerrit@5a41181]: jmx and prometheus metrics reporters - T184086 [production]
07:40 <elukey> import hue_4.8.0-2 to buster-wikimedia [production]
06:53 <marostegui> Restart dbstore* to pick up report_host - T266483 [production]
06:44 <marostegui> Restart pc1010 to pick up report_host - T266483 [production]
2020-11-09 §
22:26 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]