| 
      
        2018-05-28
      
      ยง
     | 
  
    
  | 16:31 | 
  <elukey> | 
  roll restart kafka on kafka100[1-3] to pick up new zookeeper settings | 
  [production] | 
            
  | 16:21 | 
  <elukey> | 
  zookeeper cluster restart completed (main-eqiad / conf1*) | 
  [production] | 
            
  | 16:18 | 
  <elukey> | 
  stop and mask zookeeper on conf1002 | 
  [production] | 
            
  | 16:16 | 
  <elukey> | 
  restart prometheus-burrow-exporter on kafkamon* | 
  [production] | 
            
  | 16:12 | 
  <moritzm> | 
  upgrading job runners in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 16:02 | 
  <marostegui> | 
  Stop MySQL on db2095 for testing - T190704 | 
  [production] | 
            
  | 15:59 | 
  <elukey> | 
  swap zookeeper from conf1002 to conf1005 | 
  [production] | 
            
  | 15:56 | 
  <marostegui> | 
  Reboot db1124 and db1125 for more testing - T190704 | 
  [production] | 
            
  | 15:49 | 
  <_joe_> | 
  uploading cergen 0.2.3 | 
  [production] | 
            
  | 14:44 | 
  <Amir1> | 
  EU SWAT is done | 
  [production] | 
            
  | 14:39 | 
  <ladsgroup@tin> | 
  Synchronized php-1.32.0-wmf.4/extensions/ArticlePlaceholder: Add config variable to disable SearchHookHandler (T195753) (duration: 01m 21s) | 
  [production] | 
            
  | 14:34 | 
  <ladsgroup@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Disable search integration with Article Placeholder temporarily (T195753) (duration: 01m 20s) | 
  [production] | 
            
  | 14:29 | 
  <ladsgroup@tin> | 
  Synchronized php-1.32.0-wmf.5/extensions/ArticlePlaceholder: Add config variable to disable SearchHookHandler (T195753) (duration: 01m 18s) | 
  [production] | 
            
  | 14:23 | 
  <ladsgroup@tin> | 
  Synchronized wmf-config/Wikibase-production.php: Disable Special:ItemDisambiguation in Wikidata (T195756) (duration: 01m 20s) | 
  [production] | 
            
  | 14:21 | 
  <ema> | 
  cp1045,cp2001,cp3007,cp5001: reboot with intel-microcode T127825 | 
  [production] | 
            
  | 14:00 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:434359|Enable "File mover" flag on zh.wikipedia (T195247)]] (duration: 01m 19s) | 
  [production] | 
            
  | 13:52 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:434508|New protection level on the Hungarian Wikipedia - trusted (T194568)]] (duration: 01m 20s) | 
  [production] | 
            
  | 13:40 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:435628|Revert "Revert "Enable $wgUseRCPatrol on azwiki"" (T194389)]] (duration: 01m 20s) | 
  [production] | 
            
  | 13:34 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:430124|Use uploaded HD logos for yiwikisource (T193562)]] (duration: 01m 19s) | 
  [production] | 
            
  | 13:28 | 
  <zfilipin@tin> | 
  Synchronized static/images/project-logos/: SWAT: [[gerrit:430123|Upload new logos for yiwikisource (T193562)]] (duration: 01m 19s) | 
  [production] | 
            
  | 13:21 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:435106|Enable template editor group on newiki (T195557)]] (duration: 01m 21s) | 
  [production] | 
            
  | 13:13 | 
  <volans> | 
  restarted pdfrender on scb1002 | 
  [production] | 
            
  | 13:03 | 
  <moritzm> | 
  upgrading app servers in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 12:09 | 
  <arturo> | 
  T194665 aborrero@install1002:~$ sudo -i reprepro --noskipold -C 'thirdparty/mono-project-stretch' update stretch-wikimedia | 
  [production] | 
            
  | 12:08 | 
  <arturo> | 
  T194665 aborrero@install1002:~$ sudo -i reprepro --noskipold -C 'thirdparty/mono-project-jessie' update jessie-wikimedia | 
  [production] | 
            
  | 12:05 | 
  <arturo> | 
  aborrero@install1002:~$ sudo -i reprepro --noskipold -C 'thirdparty/mono-project-trusty' update trusty-wikimedia | 
  [production] | 
            
  | 11:31 | 
  <ema> | 
  cp1008: reboot with intel-microcode T127825 | 
  [production] | 
            
  | 11:27 | 
  <moritzm> | 
  upgrading API servers in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 11:10 | 
  <marostegui> | 
  Stop MySQL on db2092 to copy its content to db2094 - T190704 | 
  [production] | 
            
  | 10:36 | 
  <moritzm> | 
  upgrading mw1266-mw1275 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 10:14 | 
  <moritzm> | 
  upgrading eqiad video scalers to hhvm-wikidiff 1.7.0 | 
  [production] | 
            
  | 10:09 | 
  <jdrewniak@tin> | 
  Synchronized portals: Wikimedia Portals Update: [[gerrit:435759|Bumping portals to master (T128546)]] (duration: 01m 20s) | 
  [production] | 
            
  | 10:08 | 
  <jdrewniak@tin> | 
  Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:435759|Bumping portals to master (T128546)]] (duration: 01m 21s) | 
  [production] | 
            
  | 09:07 | 
  <moritzm> | 
  upgrading mw1299-mw1306 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 09:04 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1083 after alter table (duration: 01m 20s) | 
  [production] | 
            
  | 09:04 | 
  <marostegui> | 
  Deploy schema change on s1 primary master (db1052) - T191519 | 
  [production] | 
            
  | 09:00 | 
  <marostegui> | 
  Deploy schema change on db1052 (s1 primary master) - T190148 | 
  [production] | 
            
  | 08:27 | 
  <moritzm> | 
  upgrading mw1221-mw1235 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 07:40 | 
  <XioNoX> | 
  enable backup tunnel routing between cr2-ulsfo and cr1-eqdfw - T195584 | 
  [production] | 
            
  | 07:31 | 
  <marostegui> | 
  Deploy schema change on db1083 - T190148 T191519 T188299 | 
  [production] | 
            
  | 07:31 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1083 for alter table (duration: 01m 20s) | 
  [production] | 
            
  | 07:29 | 
  <moritzm> | 
  upgrading mw1238-mw1258 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) | 
  [production] | 
            
  | 07:27 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1106 after alter table (duration: 01m 22s) | 
  [production] | 
            
  | 07:10 | 
  <moritzm> | 
  uploaded hhvm-wikidiff2 1.7.0 (source package name php-wikidiff2) to apt.wikimedia.org | 
  [production] | 
            
  | 06:44 | 
  <moritzm> | 
  installing ruby-loofah security updates | 
  [production] | 
            
  | 06:36 | 
  <elukey> | 
  reimage druid1003 to Debian Stretch (Analytics cluster, backend for Pivot/Turnilo) | 
  [production] | 
            
  | 05:25 | 
  <marostegui> | 
  Stop MySQL on db2075 to copy its content to db2095 - T190704 | 
  [production] | 
            
  | 05:23 | 
  <marostegui> | 
  Deploy schema change on db1106 with replication, this will generate lag on labs - T190148 T191519 T188299 | 
  [production] | 
            
  | 05:23 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1106 for alter table (duration: 01m 24s) | 
  [production] | 
            
  | 03:14 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.32.0-wmf.4) (duration: 14m 30s) | 
  [production] |