| 
      
        2018-04-24
      
      ยง
     | 
  
    
  | 16:40 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@fbce520]: Set the delete probability to 100% - T192689 | 
  [production] | 
            
  | 16:38 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1098:3316 after alter table (duration: 00m 58s) | 
  [production] | 
            
  | 16:30 | 
  <elukey> | 
  restart hadoop hdfs journalnode on analytics1035/52 to pick up prometheus jmx settings | 
  [production] | 
            
  | 16:11 | 
  <elukey> | 
  restart hadoop-hdfs-journalnode on analytics1028 to pick up prometheus monitoring | 
  [production] | 
            
  | 16:10 | 
  <bstorm_> | 
  Added views for new MCR tables on labsdb1011 (slots, slot_roles, content and content_models) | 
  [production] | 
            
  | 16:08 | 
  <marostegui> | 
  Reload haproxy on dbproxy1010 to repool labsdb1011 - https://phabricator.wikimedia.org/T184446 | 
  [production] | 
            
  | 15:59 | 
  <godog> | 
  reimage restbase1010 after ssd swap - T189822 | 
  [production] | 
            
  | 15:54 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1110 with full weight (duration: 00m 58s) | 
  [production] | 
            
  | 14:41 | 
  <elukey> | 
  restart hadoop hdfs journalnode on analytics1028 to pick up jmx settings | 
  [production] | 
            
  | 14:40 | 
  <sbisson@tin> | 
  Finished deploy [kartotherian/deploy@86da82d]: Deploy latest kartotherian with updated fallbacks and support lang=local (duration: 06m 29s) | 
  [production] | 
            
  | 14:34 | 
  <sbisson@tin> | 
  Started deploy [kartotherian/deploy@86da82d]: Deploy latest kartotherian with updated fallbacks and support lang=local | 
  [production] | 
            
  | 14:02 | 
  <Amir1> | 
  EU SWAT is done | 
  [production] | 
            
  | 14:01 | 
  <hoo@tin> | 
  Synchronized wmf-config/abusefilter.php: Grant Meta-Wiki sysops the ability to edit global abusefilter rules (T192722) (duration: 00m 59s) | 
  [production] | 
            
  | 13:58 | 
  <hoo@tin> | 
  Synchronized wmf-config/: Properly set default for $wmgWikibaseSiteGroup (T188456) (duration: 00m 58s) | 
  [production] | 
            
  | 13:56 | 
  <hoo@tin> | 
  Synchronized wmf-config/: Properly set default for $wmgWikibaseSiteGroup (T188456) (duration: 01m 00s) | 
  [production] | 
            
  | 13:43 | 
  <ladsgroup@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: [[gerrit:428623|Increase the timespan of rate limit in wikidata from 1m to 5m (T192690)]] (duration: 00m 58s) | 
  [production] | 
            
  | 13:37 | 
  <ladsgroup@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: [[gerrit:428624|Clean up old config for logging autopatrol actions (T184485)]] (duration: 00m 58s) | 
  [production] | 
            
  | 13:28 | 
  <ladsgroup@tin> | 
  Synchronized wmf-config/Wikibase-production.php: [[gerrit:425589|Add badge for good lists (T190976)]] (duration: 00m 55s) | 
  [production] | 
            
  | 13:17 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/throttle.php: SWAT: [[gerrit:428597|New throttle rule for IndigenizeWikipedia event, clean obsolete rules (T192827)]] (duration: 00m 58s) | 
  [production] | 
            
  | 13:06 | 
  <hoo@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Set default for $wmgWikibaseSiteGroup (T188456) (duration: 00m 59s) | 
  [production] | 
            
  | 12:51 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1109 (duration: 00m 58s) | 
  [production] | 
            
  | 12:31 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Increase db1110 load (duration: 00m 58s) | 
  [production] | 
            
  | 12:28 | 
  <elukey> | 
  cleanup /home/elukey/zookeeper backup files taken before the 3.4.9 migration | 
  [production] | 
            
  | 12:13 | 
  <marostegui> | 
  Deploy schema change on db1098:3316 - T191519 T188299 T190148 | 
  [production] | 
            
  | 12:11 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1098:3316 for alter table (duration: 00m 58s) | 
  [production] | 
            
  | 12:10 | 
  <elukey> | 
  reimage analytics106[34] to Debian Stretch | 
  [production] | 
            
  | 12:01 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1088 after alter table (duration: 00m 58s) | 
  [production] | 
            
  | 11:54 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1110 with low load (duration: 00m 59s) | 
  [production] | 
            
  | 11:44 | 
  <moritzm> | 
  reimaging mw1241, mw1242, mw1243 (app servers) to stretch | 
  [production] | 
            
  | 10:58 | 
  <moritzm> | 
  reimaging mw1224, mw1225, mw1226 (API servers) to stretch | 
  [production] | 
            
  | 10:50 | 
  <elukey> | 
  reimage analytics106[56] to Debian Stretch | 
  [production] | 
            
  | 10:49 | 
  <arturo> | 
  enable puppet in labtestcontrol2001 to sync with repo changes | 
  [production] | 
            
  | 10:39 | 
  <akosiaris> | 
  starting a very slow rolling reboot of all VMs on codfw ganeti cluster T150532 | 
  [production] | 
            
  | 10:39 | 
  <akosiaris> | 
  upgrade to qemu 2.8 on codfw ganeti cluster. T150532 | 
  [production] | 
            
  | 10:31 | 
  <jynus> | 
  stop and reimage db1110 | 
  [production] | 
            
  | 10:01 | 
  <apergos> | 
  reimaged snapshot1001 for testing with php7/stretch | 
  [production] | 
            
  | 09:39 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1110 (duration: 00m 58s) | 
  [production] | 
            
  | 09:28 | 
  <marostegui> | 
  Deploy schema change on db1088 - T191519 T188299 T190148 | 
  [production] | 
            
  | 09:28 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1088 for alter table (duration: 00m 58s) | 
  [production] | 
            
  | 09:25 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@1661f69]: Increase the deletion probability to 50% and expose the CSS end points, take #3 - T192689 T190846 (duration: 04m 30s) | 
  [production] | 
            
  | 09:23 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1093 after alter table (duration: 03m 06s) | 
  [production] | 
            
  | 09:21 | 
  <moritzm> | 
  reimaging mw1221, mw1222, mw1223 (API servers) to stretch | 
  [production] | 
            
  | 09:21 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@1661f69]: Increase the deletion probability to 50% and expose the CSS end points, take #3 - T192689 T190846 | 
  [production] | 
            
  | 09:21 | 
  <moritzm> | 
  reimaging mw1221, mw1222, mw1223 (app servers) to stretch | 
  [production] | 
            
  | 09:21 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@1661f69]: Increase the deletion probability to 50% and expose the CSS end points, take #2 - T192689 T190846 (duration: 03m 03s) | 
  [production] | 
            
  | 09:18 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@1661f69]: Increase the deletion probability to 50% and expose the CSS end points, take #2 - T192689 T190846 | 
  [production] | 
            
  | 09:17 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@1661f69]: Increase the deletion probability to 50% and expose the CSS end points - T192689 T190846 (duration: 13m 13s) | 
  [production] | 
            
  | 09:12 | 
  <moritzm> | 
  reimaging mw1273, mw1274, mw1275 (app servers) to stretch | 
  [production] | 
            
  | 09:03 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@1661f69]: Increase the deletion probability to 50% and expose the CSS end points - T192689 T190846 | 
  [production] | 
            
  | 08:17 | 
  <hoo> | 
  Finished running populateSitesTable.php for all wikis (T192628, T192632, T192631, T192633) | 
  [production] |