| 
      
        2018-08-23
      
      ยง
     | 
  
    
  | 12:00 | 
  <marostegui> | 
  Deploy schema change on db1105:3312 | 
  [production] | 
            
  | 12:00 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1105:3312 (duration: 00m 55s) | 
  [production] | 
            
  | 11:56 | 
  <addshore> | 
  SWAT done | 
  [production] | 
            
  | 11:50 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config: T199800 [[gerrit:Cleanup wikdiff2 mobile moved paragraph config]] (duration: 00m 55s) | 
  [production] | 
            
  | 11:40 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T202555 [[gerrit:454590|Require autoconfirmed status to edit the 828 namespace at es.wikibooks]] (duration: 00m 56s) | 
  [production] | 
            
  | 11:35 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T202065 [[gerrit:454330|Switch 'wikidata-staff' add/remove 'interface-admin' for own account]] (duration: 00m 55s) | 
  [production] | 
            
  | 11:19 | 
  <moritzm> | 
  installing shared-mime-info updates from stretch 9.5 point release | 
  [production] | 
            
  | 11:18 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: T199800 [[gerrit:454026|Enable moved paragrah detection everywhere]] (duration: 00m 55s) | 
  [production] | 
            
  | 11:01 | 
  <marostegui> | 
  Deploy schema change on dbstore1002:s2 | 
  [production] | 
            
  | 11:00 | 
  <gehel> | 
  new SSL certs / tlsproxy deployed on elastic nodes - T198351 | 
  [production] | 
            
  | 10:56 | 
  <marostegui> | 
  Deploy schema change on s2 codfw masters (db2035) with replication - this will generate lag on s2 codfw | 
  [production] | 
            
  | 10:35 | 
  <moritzm> | 
  installing openldap updates from stretch 9.5 point release | 
  [production] | 
            
  | 10:27 | 
  <akosiaris> | 
  reboot analytics-tool* hosts for the IP renumbering change T202559 | 
  [production] | 
            
  | 10:27 | 
  <akosiaris> | 
  purge rec dns caches for analytics-tool* hosts T202559 | 
  [production] | 
            
  | 10:15 | 
  <elukey> | 
  restart druid-broker on druid100[4-5] due to unresponsiveness (still unclear why) | 
  [production] | 
            
  | 10:00 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1088 (duration: 00m 54s) | 
  [production] | 
            
  | 09:59 | 
  <marostegui> | 
  Deploy schema change on s6 primary master (db1061) | 
  [production] | 
            
  | 09:50 | 
  <moritzm> | 
  installing debdeploy update | 
  [production] | 
            
  | 09:40 | 
  <moritzm> | 
  restarting etherpad-lite for nodejs security update | 
  [production] | 
            
  | 09:29 | 
  <moritzm> | 
  installing nodejs security updates on wtp* (Parsoid servers) | 
  [production] | 
            
  | 09:28 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1088 (duration: 00m 55s) | 
  [production] | 
            
  | 09:23 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1093 (duration: 00m 56s) | 
  [production] | 
            
  | 09:17 | 
  <gehel> | 
  starting to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/444610 as part of T198351, including regeneration of SSL certs. Disabling puppet on elastic* during the operation | 
  [production] | 
            
  | 08:51 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1093 (duration: 00m 55s) | 
  [production] | 
            
  | 08:45 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1085 (duration: 00m 55s) | 
  [production] | 
            
  | 08:42 | 
  <addshore> | 
  done with deploy window | 
  [production] | 
            
  | 08:41 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/Wikibase.php: T201832 Stuff for new link formatter on testwikidata [[gerrit:454510]] (duration: 00m 55s) | 
  [production] | 
            
  | 08:39 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T201832 Stuff for new link formatter on testwikidata [[gerrit:454510]] [[gerrit:452365]] [[gerrit:454764]] (duration: 00m 56s) | 
  [production] | 
            
  | 07:44 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1085 (duration: 00m 55s) | 
  [production] | 
            
  | 07:39 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1113:3316 (duration: 00m 56s) | 
  [production] | 
            
  | 07:30 | 
  <moritzm> | 
  installing openssl updates | 
  [production] | 
            
  | 07:23 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s3 on codfw (lag will be generated on codfw) T59186 | 
  [production] | 
            
  | 07:12 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s1 T59186 | 
  [production] | 
            
  | 06:47 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s7 T59186 | 
  [production] | 
            
  | 06:31 | 
  <marostegui> | 
  Enable semi-sync on es2 - T202364 | 
  [production] | 
            
  | 06:22 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1113:3316 (duration: 00m 54s) | 
  [production] | 
            
  | 06:16 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1098:3316 (duration: 00m 55s) | 
  [production] | 
            
  | 05:57 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s2 T59186 | 
  [production] | 
            
  | 05:50 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s4 T59186 | 
  [production] | 
            
  | 05:46 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s6 T59186 | 
  [production] | 
            
  | 05:19 | 
  <marostegui> | 
  Drop blob_orphans and blob_tracking from s5 T59186 | 
  [production] | 
            
  | 04:55 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1098:3316 (duration: 00m 55s) | 
  [production] | 
            
  | 04:44 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1096:3316 (duration: 00m 57s) | 
  [production] | 
            
  | 03:14 | 
  <ejegg> | 
  updated payments-wiki from 268539c2bd to 763e14b6df | 
  [production] | 
            
  | 03:06 | 
  <l10nupdate@deploy1001> | 
  ResourceLoader cache refresh completed at Thu Aug 23 03:06:25 UTC 2018 (duration 10m 22s) | 
  [production] | 
            
  | 02:56 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.18) (duration: 13m 27s) | 
  [production] | 
            
  | 02:34 | 
  <ejegg> | 
  updated CiviCRM from affd1b5e9a to 4e0dcf7cb4 | 
  [production] | 
            
  | 02:29 | 
  <ejegg> | 
  updated SmashPig standalone deploy from 3603e191a3 to cd46beefb7 | 
  [production] | 
            
  | 02:23 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.16) (duration: 09m 10s) | 
  [production] | 
            
  | 00:36 | 
  <twentyafterfour> | 
  phabricator will be down momentarily for apache restart, downtime scheduled in icinga | 
  [production] |