5251-5300 of 10000 results (80ms)
2019-10-07 ยง
17:46 <ottomata> stat1007 is unresponsive, can't login via mgmt either. powercycling. [production]
17:28 <XioNoX> add BGP route damping on IX sessions - eqiad - T222424 [production]
17:27 <XioNoX> add BGP route damping on IX sessions - esams - T222424 [production]
17:22 <XioNoX> add BGP route damping on IX sessions - eqsin - T222424 [production]
15:34 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@334e809]: Update mobileapps to 16cb9ae (duration: 06m 28s) [production]
15:30 <@> helmfile [CODFW] Ran 'apply' command on namespace 'restrouter' for release 'production' . [production]
15:29 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'restrouter' for release 'production' . [production]
15:27 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@334e809]: Update mobileapps to 16cb9ae [production]
15:27 <@> helmfile [STAGING] Ran 'apply' command on namespace 'restrouter' for release 'staging' . [production]
15:14 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Stop writing wmgVisualEditorEnableNewMobileContext (duration: 00m 51s) [production]
15:13 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: Stop reading wmgVisualEditorEnableNewMobileContext (duration: 00m 52s) [production]
14:25 <arturo> upgrading openstack in CloudVPS. Some IRC bots and related stuff may be unavailable (T212302) [production]
14:17 <marostegui> Deploy schema change on db1139:3316 - T233135 T234066 [production]
13:27 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:541249|Set all of wikidata to write both for item term store (T225055)]] (duration: 00m 54s) [production]
13:26 <mobrovac@deploy1001> Finished deploy [restbase/deploy@1337290]: Minor tweaks to VE logging, v2 (duration: 06m 38s) [production]
13:19 <mobrovac@deploy1001> Started deploy [restbase/deploy@1337290]: Minor tweaks to VE logging, v2 [production]
13:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1098:3316 schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9248 and previous config saved to /var/cache/conftool/dbconfig/20191007-131720-marostegui.json [production]
13:16 <mobrovac@deploy1001> Finished deploy [restbase/deploy@bf72f5c]: Minor tweaks to VE logging (duration: 07m 01s) [production]
13:13 <elukey> upload python-kafka and python3-kafka 1.4.7-1 to buster-wikimedia - T222941 [production]
13:09 <mobrovac@deploy1001> Started deploy [restbase/deploy@bf72f5c]: Minor tweaks to VE logging [production]
13:05 <mobrovac@deploy1001> Finished deploy [restbase/deploy@5321aac]: (no justification provided) (duration: 00m 29s) [production]
13:04 <mobrovac@deploy1001> Started deploy [restbase/deploy@5321aac]: (no justification provided) [production]
13:04 <mobrovac@deploy1001> deploy aborted: Minor tweaks to VE logging (duration: 01m 07s) [production]
13:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1096:3315 after schema change T233625', diff saved to https://phabricator.wikimedia.org/P9247 and previous config saved to /var/cache/conftool/dbconfig/20191007-130317-marostegui.json [production]
13:03 <mobrovac@deploy1001> Started deploy [restbase/deploy@fe39197]: Minor tweaks to VE logging [production]
12:54 <akosiaris@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=restrouter [production]
12:54 <elukey> upload python-kafka and python3-kafka 1.4.7-1 to stretch-wikimedia - T222941 [production]
11:44 <Lucas_WMDE> EU SWAT done [production]
11:44 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:540679|Get rid of main page hack for fixcopyrightwiki (T120085)]] (duration: 00m 52s) [production]
11:42 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:540678|Set $wgMainPageIsDomainRoot true for fixcopyrightwiki (T120085)]] (duration: 00m 52s) [production]
11:41 <Amir1> another hack bites the dust [production]
11:35 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.34.0-wmf.25/extensions/GrowthExperiments/: SWAT: [[gerrit:541139|Homepage: Don't use flexbox for vertical layouts in mobile start module (T234380)]] (duration: 00m 53s) [production]
11:18 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:541008|Enable partial blocks on nlwiki (T234685)]] (duration: 00m 52s) [production]
11:16 <arturo> added bdsync 0.11.1-1~wmf1 to buster-wikimedia (T234683) [production]
10:59 <mobrovac@deploy1001> Finished deploy [restbase/deploy@5321aac]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #5 (duration: 04m 17s) [production]
10:55 <mobrovac@deploy1001> Started deploy [restbase/deploy@5321aac]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #5 [production]
10:54 <mobrovac@deploy1001> Finished deploy [restbase/deploy@5321aac]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #4 (duration: 04m 27s) [production]
10:50 <mobrovac@deploy1001> Started deploy [restbase/deploy@5321aac]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #4 [production]
10:48 <mobrovac@deploy1001> Finished deploy [restbase/deploy@5321aac]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #3 (duration: 03m 53s) [production]
10:44 <mobrovac@deploy1001> Started deploy [restbase/deploy@5321aac]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #3 [production]
10:37 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:541217| Bumping portals to master (T128546)]] (duration: 00m 51s) [production]
10:36 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:541217| Bumping portals to master (T128546)]] (duration: 00m 53s) [production]
10:31 <_joe_> uploading confd 0.16.0 to stretch [production]
10:21 <mobrovac@deploy1001> Finished deploy [restbase/deploy@1798e39]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #2 (duration: 01m 56s) [production]
10:19 <mobrovac@deploy1001> Started deploy [restbase/deploy@1798e39]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests, take #2 [production]
10:16 <mobrovac@deploy1001> Finished deploy [restbase/deploy@1798e39]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests - T233127 T234772 (duration: 05m 58s) [production]
10:10 <mobrovac@deploy1001> Started deploy [restbase/deploy@1798e39]: Skip checking resources on start-up, add banwiki, add metrics/mediarequests end points and log all VE requests - T233127 T234772 [production]
09:55 <marostegui> Deploy schema change on db2129 (s6 codfw master), this will generate lag on s6 codfw - T233135 T234066 [production]
08:34 <hashar> gerrit: force reindexing all changes ( gerrit index start changes --force ) [production]
07:09 <marostegui> Remove grants for dbproxy1006 on m1 databases - T231280 [production]