| 2018-05-31
      
      § | 
    
  | 10:45 | <elukey> | removed Pivot from thorium (pivot.wikimedia.org now simply redirects to Turnilo) | [production] | 
            
  | 09:45 | <marostegui> | Stop MySQL on db2062 to clone db2092 | [production] | 
            
  | 09:45 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Depool db2062 for cloning db2092 (duration: 01m 22s) | [production] | 
            
  | 09:40 | <volans> | restarting Icinga, issues processing the command file | [production] | 
            
  | 09:35 | <jynus> | testing icinga alerting | [production] | 
            
  | 09:28 | <elukey> | reimage druid1006 to Debian Stretch | [production] | 
            
  | 09:11 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1082 fully (duration: 01m 21s) | [production] | 
            
  | 08:42 | <gehel> | power off elastic2018 - T196045 | [production] | 
            
  | 08:39 | <Amir1> | ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=commonswiki --sleep 2 --check-old | [production] | 
            
  | 08:09 | <gehel> | power reset elastic2018 | [production] | 
            
  | 08:08 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=inactive; selector: dc=codfw,service=mathoid,cluster=scb,name=scb.* | [production] | 
            
  | 08:07 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=inactive; selector: dc=eqiad,service=mathoid,cluster=scb,name=scb.* | [production] | 
            
  | 07:54 | <akosiaris> | reimage kubernetes1004 without swap | [production] | 
            
  | 07:48 | <ema> | power-cycle elastic2018 | [production] | 
            
  | 07:41 | <marostegui> | Stop Replication on db2066 | [production] | 
            
  | 07:40 | <akosiaris> | re-enable puppet across the fleet | [production] | 
            
  | 07:30 | <akosiaris> | disable puppet for https://gerrit.wikimedia.org/r/#/c/436468/ merge cross fleet | [production] | 
            
  | 07:27 | <marostegui> | Deploy schema change on s5 codfw master (db2052) this will generate lag on codfw  - T191316 T192926 T89737 T195193 | [production] | 
            
  | 07:24 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Repool db2066 after alter table (duration: 01m 22s) | [production] | 
            
  | 06:38 | <marostegui> | Deploy schema change on db2066 - T191316 T192926 T89737 T195193 | [production] | 
            
  | 06:38 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Depool db2066 for alter table (duration: 01m 21s) | [production] | 
            
  | 06:30 | <marostegui> | Deploy schema change on db2092:3315 and db2094:3315 - T191316 T192926 T89737 T195193 | [production] | 
            
  | 06:28 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Repool db2084:3315 after alter table (duration: 01m 28s) | [production] | 
            
  | 06:09 | <akosiaris> | reimage ganeti1003, ganeti1007 to stretch | [production] | 
            
  | 05:59 | <elukey> | reimage druid1005 to Debian Stretch | [production] | 
            
  | 05:51 | <elukey> | delete /tmp/scap_l10n_1501525840,scap_l10n_1501525840,l10nstuff,l10nstuff3 from tin to free some space in the root partition (1.9G left) | [production] | 
            
  | 05:40 | <elukey> | restart pdfrender on scb1001 | [production] | 
            
  | 05:25 | <marostegui> | Deploy schema change on db2084:3315 - T191316 T192926 T89737 T195193 | [production] | 
            
  | 05:24 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Depool db2084:3315 for alter table (duration: 01m 27s) | [production] | 
            
  | 05:13 | <moritzm> | installing python-crypto security updates on trusty | [production] | 
            
  | 04:19 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu May 31 04:19:57 UTC 2018 (duration 14m 48s) | [production] | 
            
  | 04:18 | <moritzm> | rebooting wtp2001/wtp2015 for microcode updates | [production] | 
            
  | 04:05 | <l10nupdate@tin> | scap sync-l10n completed (1.32.0-wmf.6) (duration: 18m 15s) | [production] | 
            
  | 03:03 | <l10nupdate@tin> | scap sync-l10n completed (1.32.0-wmf.5) (duration: 16m 18s) | [production] | 
            
  | 02:09 | <pnorman@tin> | Finished deploy [tilerator/deploy@2a26f1e] (cleartables): Deploy style with fewer fonts (duration: 00m 24s) | [production] | 
            
  | 02:08 | <pnorman@tin> | Started deploy [tilerator/deploy@2a26f1e] (cleartables): Deploy style with fewer fonts | [production] | 
            
  | 02:02 | <pnorman@tin> | Finished deploy [tilerator/deploy@78448de] (cleartables): Deploy style with fewer fonts (duration: 00m 25s) | [production] | 
            
  | 02:01 | <pnorman@tin> | Started deploy [tilerator/deploy@78448de] (cleartables): Deploy style with fewer fonts | [production] | 
            
  | 01:39 | <pnorman@tin> | Finished deploy [tilerator/deploy@fad9969] (cleartables): Deploy updated stylesheet (duration: 00m 24s) | [production] | 
            
  | 01:39 | <pnorman@tin> | Started deploy [tilerator/deploy@fad9969] (cleartables): Deploy updated stylesheet | [production] | 
            
  | 01:38 | <pnorman@tin> | Finished deploy [tilerator/deploy@78d1b82] (cleartables): Deploy updated stylesheet (duration: 00m 05s) | [production] | 
            
  | 01:38 | <pnorman@tin> | Started deploy [tilerator/deploy@78d1b82] (cleartables): Deploy updated stylesheet | [production] | 
            
  | 01:30 | <pnorman@tin> | Finished deploy [tilerator/deploy@78d1b82] (cleartables): Deploy updated stylesheet (duration: 00m 25s) | [production] | 
            
  | 01:30 | <pnorman@tin> | Started deploy [tilerator/deploy@78d1b82] (cleartables): Deploy updated stylesheet | [production] | 
            
  | 01:05 | <mutante> | $lang.planet.wikimedia.org is changing software from planet-venus to rawdog | [production] |