| 
      
        2018-07-18
      
      §
     | 
  
    
  | 12:24 | 
  <elukey> | 
  manually added queued.max.messages.kbytes: 65535 to eventstreams on scb2002 as test for T199813 | 
  [production] | 
            
  | 12:16 | 
  <moritzm> | 
  installing PHP security updates on yubiauth servers | 
  [production] | 
            
  | 12:02 | 
  <moritzm> | 
  installing PHP security update on bohrium | 
  [production] | 
            
  | 11:35 | 
  <moritzm> | 
  installing imagemagick security updates | 
  [production] | 
            
  | 11:34 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 11:32 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/abusefilter.php: SWAT: [[gerrit:446539|Enable AbuseFilter block option on itwiktionary (T199783)]] (duration: 00m 54s) | 
  [production] | 
            
  | 11:25 | 
  <moritzm> | 
  installing policykit security updates on trusty | 
  [production] | 
            
  | 11:20 | 
  <Amir1> | 
  making ores tables on euwiki | 
  [production] | 
            
  | 11:15 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:444664|Create Publisher namespace in Bengali Wikisource (T199028)]] (duration: 00m 54s) | 
  [production] | 
            
  | 11:05 | 
  <ladsgroup@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:446563|Enable ores extension wp10 storage in Basque Wikipedia (T198358)]] (duration: 00m 54s) | 
  [production] | 
            
  | 10:30 | 
  <moritzm> | 
  ran puppet clean/deactivate for lawrencium, long-standing source of errors in package deployments (T191360) | 
  [production] | 
            
  | 10:24 | 
  <moritzm> | 
  installing cups/libcups security updates for jessie | 
  [production] | 
            
  | 10:10 | 
  <dcausse> | 
  T156137: banning elastic1030 from the cluster (high load) | 
  [production] | 
            
  | 10:02 | 
  <dcausse> | 
  clearing internal cache on elastic1030 | 
  [production] | 
            
  | 09:58 | 
  <hashar> | 
  contint1001 dropping unused docker containers | 
  [production] | 
            
  | 09:36 | 
  <moritzm> | 
  draining restbase1007 for eventual reboot for tests with SSBD microcode | 
  [production] | 
            
  | 08:51 | 
  <godog> | 
  run xfs_repair on filesystems reporting negative space available on ms-be1043 - T199198 | 
  [production] | 
            
  | 08:50 | 
  <moritzm> | 
  reboot ms-be1040 for tests with SSBD microcode | 
  [production] | 
            
  | 08:45 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1085 after alter table (duration: 00m 53s) | 
  [production] | 
            
  | 08:38 | 
  <twentyafterfour> | 
  restart apache on phab1001 | 
  [production] | 
            
  | 08:37 | 
  <chasemp> | 
  deploy antivandalism stuff to phab https://gerrit.wikimedia.org/r/c/operations/puppet/+/445329 (needs herald to fully enable) | 
  [production] | 
            
  | 08:31 | 
  <elukey> | 
  drain + reboot analytics1030 for kernel updates | 
  [production] | 
            
  | 08:29 | 
  <marostegui> | 
  Deploy schema change on db1085 with replication, this will generate lag on labsdb hosts for s6 T144010 T51190 T199368 | 
  [production] | 
            
  | 08:29 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1085 for alter table (duration: 01m 01s) | 
  [production] | 
            
  | 08:19 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1088 after alter table (duration: 00m 53s) | 
  [production] | 
            
  | 08:06 | 
  <marostegui> | 
  Deploy schema change on db1088 T144010 T51190 T199368 | 
  [production] | 
            
  | 08:06 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1088 for alter table (duration: 00m 53s) | 
  [production] | 
            
  | 07:41 | 
  <jforrester@deploy1001> | 
  Synchronized wmf-config/missing.php: Fix typo in missing.php Ia3aae912d9m (duration: 00m 54s) | 
  [production] | 
            
  | 07:37 | 
  <moritzm> | 
  upgrading tor on radium to 0.3.3.9 | 
  [production] | 
            
  | 07:25 | 
  <marostegui> | 
  Deploy schema change on db1052 - https://phabricator.wikimedia.org/T191316 https://phabricator.wikimedia.org/T192926 https://phabricator.wikimedia.org/T89737 https://phabricator.wikimedia.org/T195193 | 
  [production] | 
            
  | 07:22 | 
  <moritzm> | 
  updated tor on apt.wikimedia.org to 0.3.3.9 | 
  [production] | 
            
  | 06:45 | 
  <jynus> | 
  enable semi-sync on db1099:3311 and db1105:3311 | 
  [production] | 
            
  | 06:08 | 
  <marostegui> | 
  s1 failover finished T197069 | 
  [production] | 
            
  | 06:07 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: read only OFF after failover T197069  (duration: 00m 53s) | 
  [production] | 
            
  | 06:04 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Set db1067 as master T197069  (duration: 00m 53s) | 
  [production] | 
            
  | 06:01 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Set s1 on ready only for maintenance T197069  (duration: 01m 08s) | 
  [production] | 
            
  | 06:00 | 
  <marostegui> | 
  Starting s1 failover from db1052 to db1067 - T197069 | 
  [production] | 
            
  | 05:11 | 
  <Krinkle> | 
  mwscript deleteEqualMessages.php --wiki dewiki (1 page) - T45917 | 
  [production] | 
            
  | 05:11 | 
  <Krinkle> | 
  mwscript deleteEqualMessages.php --wiki ruwiki (4 pages) - T45917 | 
  [production] | 
            
  | 05:11 | 
  <Krinkle> | 
  mwscript deleteEqualMessages.php --wiki frwiki (5 pages) - T45917 | 
  [production] | 
            
  | 05:09 | 
  <Krinkle> | 
  mwscript deleteEqualMessages.php --wiki metawiki (deleted 1 page) - T45917 | 
  [production] | 
            
  | 05:09 | 
  <Krinkle> | 
  mwscript deleteEqualMessages.php --wiki test2wiki (deleted 2 pages) - T45917 | 
  [production] | 
            
  | 04:56 | 
  <marostegui> | 
  Starting s1 failover pre steps - https://phabricator.wikimedia.org/T197069 | 
  [production] | 
            
  | 03:05 | 
  <l10nupdate@deploy1001> | 
  ResourceLoader cache refresh completed at Wed Jul 18 03:05:46 UTC 2018 (duration 10m 27s) | 
  [production] | 
            
  | 02:55 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.13) (duration: 09m 43s) | 
  [production] | 
            
  | 02:28 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.12) (duration: 08m 35s) | 
  [production] | 
            
  | 00:00 | 
  <reedy@deploy1001> | 
  Finished scap: log event fixes (duration: 33m 18s) | 
  [production] |