| 2018-07-18
      
      § | 
    
  | 11:32 | <zfilipin@deploy1001> | Synchronized wmf-config/abusefilter.php: SWAT: [[gerrit:446539|Enable AbuseFilter block option on itwiktionary (T199783)]] (duration: 00m 54s) | [production] | 
            
  | 11:25 | <moritzm> | installing policykit security updates on trusty | [production] | 
            
  | 11:20 | <Amir1> | making ores tables on euwiki | [production] | 
            
  | 11:15 | <zfilipin@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:444664|Create Publisher namespace in Bengali Wikisource (T199028)]] (duration: 00m 54s) | [production] | 
            
  | 11:05 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:446563|Enable ores extension wp10 storage in Basque Wikipedia (T198358)]] (duration: 00m 54s) | [production] | 
            
  | 10:30 | <moritzm> | ran puppet clean/deactivate for lawrencium, long-standing source of errors in package deployments (T191360) | [production] | 
            
  | 10:24 | <moritzm> | installing cups/libcups security updates for jessie | [production] | 
            
  | 10:10 | <dcausse> | T156137: banning elastic1030 from the cluster (high load) | [production] | 
            
  | 10:02 | <dcausse> | clearing internal cache on elastic1030 | [production] | 
            
  | 09:58 | <hashar> | contint1001 dropping unused docker containers | [production] | 
            
  | 09:36 | <moritzm> | draining restbase1007 for eventual reboot for tests with SSBD microcode | [production] | 
            
  | 08:51 | <godog> | run xfs_repair on filesystems reporting negative space available on ms-be1043 - T199198 | [production] | 
            
  | 08:50 | <moritzm> | reboot ms-be1040 for tests with SSBD microcode | [production] | 
            
  | 08:45 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1085 after alter table (duration: 00m 53s) | [production] | 
            
  | 08:38 | <twentyafterfour> | restart apache on phab1001 | [production] | 
            
  | 08:37 | <chasemp> | deploy antivandalism stuff to phab https://gerrit.wikimedia.org/r/c/operations/puppet/+/445329 (needs herald to fully enable) | [production] | 
            
  | 08:31 | <elukey> | drain + reboot analytics1030 for kernel updates | [production] | 
            
  | 08:29 | <marostegui> | Deploy schema change on db1085 with replication, this will generate lag on labsdb hosts for s6 T144010 T51190 T199368 | [production] | 
            
  | 08:29 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1085 for alter table (duration: 01m 01s) | [production] | 
            
  | 08:19 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1088 after alter table (duration: 00m 53s) | [production] | 
            
  | 08:06 | <marostegui> | Deploy schema change on db1088 T144010 T51190 T199368 | [production] | 
            
  | 08:06 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1088 for alter table (duration: 00m 53s) | [production] | 
            
  | 07:41 | <jforrester@deploy1001> | Synchronized wmf-config/missing.php: Fix typo in missing.php Ia3aae912d9m (duration: 00m 54s) | [production] | 
            
  | 07:37 | <moritzm> | upgrading tor on radium to 0.3.3.9 | [production] | 
            
  | 07:25 | <marostegui> | Deploy schema change on db1052 - https://phabricator.wikimedia.org/T191316 https://phabricator.wikimedia.org/T192926 https://phabricator.wikimedia.org/T89737 https://phabricator.wikimedia.org/T195193 | [production] | 
            
  | 07:22 | <moritzm> | updated tor on apt.wikimedia.org to 0.3.3.9 | [production] | 
            
  | 06:45 | <jynus> | enable semi-sync on db1099:3311 and db1105:3311 | [production] | 
            
  | 06:08 | <marostegui> | s1 failover finished T197069 | [production] | 
            
  | 06:07 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: read only OFF after failover T197069  (duration: 00m 53s) | [production] | 
            
  | 06:04 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Set db1067 as master T197069  (duration: 00m 53s) | [production] | 
            
  | 06:01 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Set s1 on ready only for maintenance T197069  (duration: 01m 08s) | [production] | 
            
  | 06:00 | <marostegui> | Starting s1 failover from db1052 to db1067 - T197069 | [production] | 
            
  | 05:11 | <Krinkle> | mwscript deleteEqualMessages.php --wiki dewiki (1 page) - T45917 | [production] | 
            
  | 05:11 | <Krinkle> | mwscript deleteEqualMessages.php --wiki ruwiki (4 pages) - T45917 | [production] | 
            
  | 05:11 | <Krinkle> | mwscript deleteEqualMessages.php --wiki frwiki (5 pages) - T45917 | [production] | 
            
  | 05:09 | <Krinkle> | mwscript deleteEqualMessages.php --wiki metawiki (deleted 1 page) - T45917 | [production] | 
            
  | 05:09 | <Krinkle> | mwscript deleteEqualMessages.php --wiki test2wiki (deleted 2 pages) - T45917 | [production] | 
            
  | 04:56 | <marostegui> | Starting s1 failover pre steps - https://phabricator.wikimedia.org/T197069 | [production] | 
            
  | 03:05 | <l10nupdate@deploy1001> | ResourceLoader cache refresh completed at Wed Jul 18 03:05:46 UTC 2018 (duration 10m 27s) | [production] | 
            
  | 02:55 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.13) (duration: 09m 43s) | [production] | 
            
  | 02:28 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.12) (duration: 08m 35s) | [production] | 
            
  | 00:00 | <reedy@deploy1001> | Finished scap: log event fixes (duration: 33m 18s) | [production] | 
            
  
    | 2018-07-17
      
      § | 
    
  | 23:26 | <reedy@deploy1001> | Started scap: log event fixes | [production] | 
            
  | 22:43 | <reedy@deploy1001> | Synchronized wmf-config/interwiki-labs.php: (no justification provided) (duration: 00m 53s) | [production] | 
            
  | 22:24 | <reedy@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Remove override for foundationwiki in wgMobileUrlTemplate (duration: 00m 53s) | [production] | 
            
  | 22:21 | <reedy@deploy1001> | Synchronized tests/urls.txt: testding! (duration: 00m 53s) | [production] | 
            
  | 20:57 | <reedy@deploy1001> | Synchronized wmf-config/: make foundation.wikimedia.org writeable T188776 (duration: 00m 53s) | [production] | 
            
  | 20:50 | <reedy@deploy1001> | Finished scap: updating l10n cache for foundationwiki url changes (duration: 58m 32s) | [production] | 
            
  | 20:17 | <ebernhardson> | un-ban elastic1030 from eqiad search cluster | [production] | 
            
  | 19:52 | <reedy@deploy1001> | Started scap: updating l10n cache for foundationwiki url changes | [production] |