| 2017-11-07
      
      § | 
    
  | 23:57 | <urandom> | Decommissioning restbase2001-b.codfw.wmnet (T179422) | [production] | 
            
  | 22:50 | <ejegg> | re-enabled thank you mailer | [production] | 
            
  | 22:49 | <ejegg> | updated CiviCRM  from 85e89c657bbfcaffc42f66a4f26aa6e9686a989e to dc1b2798a051019177de3adf5f7e93fb95148ccf | [production] | 
            
  | 22:39 | <hoo> | Reset a global account's email, per T179950 | [production] | 
            
  | 22:28 | <ejegg> | updated CiviCRM from 122ba657ba347c02a3f5e3c689bb5acbce6e7364 to 85e89c657bbfcaffc42f66a4f26aa6e9686a989e | [production] | 
            
  | 21:56 | <ejegg> | updated CiviCRM from dc9d054169a196ab14f6a4070e871c2dd8580ad4 to 122ba657ba347c02a3f5e3c689bb5acbce6e7364 | [production] | 
            
  | 21:20 | <addshore@tin> | Synchronized wmf-config/Wikibase-buildentry.php: T176948 [[gerrit:389691|Remove Shared Cache settings from Wikibase-buildentry]] (duration: 00m 50s) | [production] | 
            
  | 21:16 | <addshore@tin> | Synchronized wmf-config/Wikibase.php: T176948 [[gerrit:381371|Stop using wgWikibaseSharedCacheKeyPrefix from Wikidata build]] (duration: 00m 49s) | [production] | 
            
  | 21:09 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: T176948 [[gerrit:389690|Remove wmgWikibaseUseConfigFromWikidataBuild]] PT 3/3 (LABS) (duration: 00m 50s) | [production] | 
            
  | 21:07 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T176948 [[gerrit:389690|Remove wmgWikibaseUseConfigFromWikidataBuild]] PT 2/3 (duration: 00m 52s) | [production] | 
            
  | 21:06 | <addshore@tin> | Synchronized wmf-config/Wikibase.php: T176948 [[gerrit:389690|Remove wmgWikibaseUseConfigFromWikidataBuild]] PT 1/3 (duration: 00m 50s) | [production] | 
            
  | 21:02 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T176948 [[gerrit:389689|wmgWikibaseUseConfigFromWikidataBuild flase for all of PROD]] (duration: 00m 49s) | [production] | 
            
  | 21:01 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T176948 [[gerrit:389688|Load wikibase build from mediawiki-config for wikidataclient]] (duration: 00m 50s) | [production] | 
            
  | 20:59 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T176948 [[gerrit:389687|Load wikibase build from mediawiki-config for group0 & group1]] (duration: 00m 50s) | [production] | 
            
  | 20:54 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T176948 [[gerrit:389686|Load wikibase build from mediawiki-config for hewiki]] (duration: 00m 49s) | [production] | 
            
  | 20:51 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T176948 [[gerrit:389685|Load wikibase build from mediawiki-config for wikidatawiki]] (duration: 00m 50s) | [production] | 
            
  | 20:48 | <herron> | puppet issue cleared after reverting 386666.  restarting ircecho on einsteinium | [production] | 
            
  | 20:45 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: T176948 [[gerrit:381195|#1]] [[gerrit:381199|#2]] [[gerrit:389684|#3]] Load wikibase build from mediawiki-config for BETA ONLY (duration: 00m 50s) | [production] | 
            
  | 20:28 | <addshore@tin> | Synchronized wmf-config/Wikibase.php: [[gerrit:381194|Add loading of wikibase extensions from build]] PT 3/3 (duration: 00m 50s) | [production] | 
            
  | 20:27 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: [[gerrit:381194|Add loading of wikibase extensions from build]] PT 2/3 (duration: 00m 50s) | [production] | 
            
  | 20:25 | <addshore@tin> | Synchronized wmf-config/Wikibase-buildentry.php: [[gerrit:381194|Add loading of wikibase extensions from build]] PT 1/3 (duration: 00m 49s) | [production] | 
            
  | 20:20 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: [[gerrit:381193|Remove unused wmgUseWikibasePropertySuggester]] PT 2/2 (duration: 00m 50s) | [production] | 
            
  | 20:19 | <addshore@tin> | Synchronized wmf-config/CommonSettings.php: [[gerrit:381193|Remove unused wmgUseWikibasePropertySuggester]] PT 1/2 (duration: 00m 50s) | [production] | 
            
  | 20:06 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.7 | [production] | 
            
  | 19:44 | <volans> | restarted apache2 on rhodium (puppet master failing) | [production] | 
            
  | 19:31 | <herron> | restart apache2 service on rhodium | [production] | 
            
  | 19:12 | <demon@tin> | Finished scap: wmf.7 bootstrap (duration: 48m 15s) | [production] | 
            
  | 19:03 | <awight> | begin stress test on ores* | [production] | 
            
  | 18:58 | <awight@tin> | Finished deploy [ores/deploy@29905e5]: test deployment to repair ores1009 (non-production) (duration: 00m 31s) | [production] | 
            
  | 18:57 | <awight@tin> | Started deploy [ores/deploy@29905e5]: test deployment to repair ores1009 (non-production) | [production] | 
            
  | 18:56 | <awight@tin> | Finished deploy [ores/deploy@29905e5]: test deployment to repair ores1008 (non-production) (duration: 00m 32s) | [production] | 
            
  | 18:56 | <awight@tin> | Started deploy [ores/deploy@29905e5]: test deployment to repair ores1008 (non-production) | [production] | 
            
  | 18:50 | <awight@tin> | Finished deploy [ores/deploy@29905e5]: test deployment to repair ores1002 (non-production) (duration: 00m 33s) | [production] | 
            
  | 18:49 | <awight@tin> | Started deploy [ores/deploy@29905e5]: test deployment to repair ores1002 (non-production) | [production] | 
            
  | 18:39 | <volans> | slowly running puppet on failed hosts with cumin (concurrency=5) | [production] | 
            
  | 18:38 | <awight@tin> | Finished deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) (duration: 00m 20s) | [production] | 
            
  | 18:38 | <awight@tin> | Started deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) | [production] | 
            
  | 18:36 | <_joe_> | restarting apache2 on rhodium | [production] | 
            
  | 18:24 | <demon@tin> | Started scap: wmf.7 bootstrap | [production] | 
            
  | 18:16 | <awight@tin> | Finished deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) (duration: 01m 04s) | [production] | 
            
  | 18:15 | <awight@tin> | Started deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) | [production] | 
            
  | 18:06 | <herron> | restarting puppetdb service on nitrogen | [production] | 
            
  | 17:56 | <urandom> | Decommissioning restbase2001-a.codfw.wmnet (T179422) | [production] | 
            
  | 17:55 | <elukey> | stop ircecho on einstenium (puppet shower from nitrogen) | [production] | 
            
  | 17:40 | <ema> | stop cache_text/upload rolling reboots, resuming tomorrow | [production] | 
            
  | 17:38 | <urandom> | Restart Cassandra, restbase1010-{a,b,c}.eqiad.wmnet (T178177) | [production] |