| 2018-08-30
      
      § | 
    
  | 12:59 | <elukey> | drain + reboot analytics10[28-79]* for kernel updates (will take multiple days) | [production] | 
            
  | 12:28 | <godog> | roll restart thumbor in eqiad to upgrade to 2.1 | [production] | 
            
  | 12:25 | <reedy@deploy1001> | Synchronized wmf-config/interwiki.php: Updating interwiki cache (duration: 02m 36s) | [production] | 
            
  | 12:23 | <reedy@deploy1001> | Synchronized dblists/: fixcopyrightwiki (duration: 00m 56s) | [production] | 
            
  | 12:22 | <moritzm> | uploaded grafana 4.6.4 to apt.wikimedia.org | [production] | 
            
  | 12:13 | <reedy@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: fixcopyrightwiki (duration: 00m 56s) | [production] | 
            
  | 12:11 | <reedy@deploy1001> | Synchronized static/images/project-logos: fixcopyrightwiki (duration: 00m 55s) | [production] | 
            
  | 12:10 | <reedy@deploy1001> | rebuilt and synchronized wikiversions files: fixcopyright | [production] | 
            
  | 12:07 | <reedy@deploy1001> | Synchronized dblists/: fixcopyrightwiki (duration: 00m 56s) | [production] | 
            
  | 11:41 | <banyek@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 55s) | [production] | 
            
  | 11:28 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 11:26 | <zfilipin@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:456358|Test Wikidata: Use new item ID formatter for all the items (T203147)]] (duration: 00m 53s) | [production] | 
            
  | 11:14 | <zfilipin@deploy1001> | Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:456357|Beta Wikidata: Use new item ID formatter for all the items (T203147)]] (duration: 00m 57s) | [production] | 
            
  | 11:09 | <tgr@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:455561|Add editsitejson to everyone who has editinterface (T190015)]] (duration: 01m 00s) | [production] | 
            
  | 10:37 | <moritzm> | upgrading deployment-prep to wikidiff 1.7.3 | [production] | 
            
  | 10:04 | <godog> | roll-restart thumbor to upgrade 2.1 | [production] | 
            
  | 08:52 | <godog> | roll-restart swift-proxy to send requests to thumbor in eqiad and codfw - T201858 | [production] | 
            
  | 07:34 | <banyek@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2070 (duration: 00m 57s) | [production] | 
            
  | 06:46 | <jynus> | upgrade and restart db1090 | [production] | 
            
  | 06:43 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1090 (s2 and s7) (duration: 00m 56s) | [production] | 
            
  | 05:52 | <jynus> | upgrade and restart db1119 | [production] | 
            
  | 05:49 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1119 (duration: 00m 58s) | [production] | 
            
  | 03:42 | <mutante> | running manual update of en.planet feeds (T203055) | [production] | 
            
  | 03:21 | <l10nupdate@deploy1001> | ResourceLoader cache refresh completed at Thu Aug 30 03:21:55 UTC 2018 (duration 10m 14s) | [production] | 
            
  | 03:11 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.19) (duration: 15m 12s) | [production] | 
            
  | 02:36 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.18) (duration: 15m 20s) | [production] | 
            
  
    | 2018-08-29
      
      § | 
    
  | 23:56 | <catrope@deploy1001> | Synchronized php-1.32.0-wmf.18/extensions/Translate/stringmangler/StringMatcher.php: T202058 (duration: 00m 56s) | [production] | 
            
  | 23:55 | <catrope@deploy1001> | Synchronized php-1.32.0-wmf.19/extensions/Translate/stringmangler/StringMatcher.php: T202058 (duration: 00m 59s) | [production] | 
            
  | 19:47 | <bstorm_> | failed tools and cloud vps storage services over to labstore1004 | [production] | 
            
  | 19:24 | <dduvall@deploy1001> | rebuilt and synchronized wikiversions files: group1 to 1.32.0-wmf.19 | [production] | 
            
  | 19:21 | <marxarelli> | Deploying 1.32.0-wmf.19 to group1 | [production] | 
            
  | 18:55 | <mutante> | puppet deploy of new cluster apache site inclusion for fixcopyright.wm, tested on mwdebug100*,mw1269, apache-fast-test (T202819) (gerrit:456194) | [production] | 
            
  | 18:31 | <ottomata> | debdeploy git-fat update for all nodes - T202100 | [production] | 
            
  | 18:10 | <mutante> | puppet re-enabled on mw* via cumin before last log message (cumin expected to log to SAL automatically?) | [production] | 
            
  | 18:08 | <mutante> | new apache config in sites-available for new site fixcopyright.wm is being generated by puppet on cluster, but not enabled yet (T202819) | [production] | 
            
  | 17:51 | <imarlier@deploy1001> | Finished deploy [performance/coal@7457c86]: (no justification provided) (duration: 00m 06s) | [production] | 
            
  | 17:51 | <imarlier@deploy1001> | Started deploy [performance/coal@7457c86]: (no justification provided) | [production] | 
            
  | 17:51 | <mutante> | disabling puppet on mw hosts as a precaution before deploying apache change | [production] | 
            
  | 16:49 | <marostegui> | Force RAID controller to WB policy T202051 | [production] | 
            
  | 16:21 | <rxy> | tgr@deploy1001 Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:456103|Wikidata: Use new item ID formatter for Q1-Q10000 (T201834)]] (duration: 00m 56s)  (by logmsgbot) | [production] | 
            
  | 16:21 | <rxy> | T202323 labstore1005 reboot (by arturo) | [production] | 
            
  | 16:02 | <arturo> | T202323 failover drbd primary from labstore1005 to labstore1004 | [production] | 
            
  | 15:53 | <arturo> | T202323 labstore1004 now running latest kernel | [production] | 
            
  | 15:51 | <marlier> | deploy coal | [production] | 
            
  | 15:51 | <imarlier@deploy1001> | Finished deploy [performance/coal@5d995a3]: (no justification provided) (duration: 00m 06s) | [production] | 
            
  | 15:51 | <imarlier@deploy1001> | Started deploy [performance/coal@5d995a3]: (no justification provided) | [production] | 
            
  | 15:48 | <elukey> | roll restart aqs on aqs100[4-9] to pick up the new druid config settings | [production] | 
            
  | 15:47 | <arturo> | T202323 labstore1004 reboot | [production] | 
            
  | 15:37 | <joal@deploy1001> | Finished deploy [analytics/aqs/deploy@c33f6e5]: Update wikistats2 endpoints to use user_text and page_title (duration: 11m 38s) | [production] | 
            
  | 15:26 | <joal@deploy1001> | Started deploy [analytics/aqs/deploy@c33f6e5]: Update wikistats2 endpoints to use user_text and page_title | [production] |