| 2017-11-09
      
      § | 
    
  | 18:04 | <moritzm> | rolling restart of parsoid servers in codfw for 4.9.51 kernel update | [production] | 
            
  | 17:14 | <urandom> | Restarting Cassandra, restbase2005-b.codfw.wmnet (T179419) | [production] | 
            
  | 16:33 | <urandom> | Restarting Cassandra, restbase2005-a.codfw.wmnet (T179419) | [production] | 
            
  | 15:12 | <urandom> | Creating mathoid schema (T179419) | [production] | 
            
  | 15:05 | <addshore@tin> | Synchronized wmf-config/CommonSettings-labs.php: [[gerrit:379715|Enable AdvancedSearch on beta]] LABS / BETA ONLY PT2/2 (duration: 00m 49s) | [production] | 
            
  | 15:04 | <addshore> | last sync was actually "Enable AdvancedSearch on beta" | [production] | 
            
  | 15:04 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: [[gerrit:379715|Add AdvancedSearch to extension-list-labs]] LABS / BETA ONLY PT1/2 (duration: 00m 50s) | [production] | 
            
  | 14:57 | <addshore@tin> | Synchronized wmf-config/extension-list-labs: [[gerrit:379714|Add AdvancedSearch to extension-list-labs]] LABS / BETA ONLY (duration: 00m 50s) | [production] | 
            
  | 14:42 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 14:40 | <zfilipin@tin> | Synchronized wmf-config/StartProfiler.php: SWAT: [[gerrit:390073|xenon: encode the request method as a virtual stack frame]] (duration: 00m 50s) | [production] | 
            
  | 14:35 | <ladsgroup@tin> | Synchronized wmf-config/InitialiseSettings.php: Use a threshold that ores in frwiki can stand (T180115) (duration: 00m 50s) | [production] | 
            
  | 14:30 | <zfilipin@tin> | Synchronized php-1.31.0-wmf.7/extensions/ContentTranslation/modules/: SWAT: [[gerrit:390206|Bring back the overlay support for a specific screen region (T179997)]] (duration: 00m 50s) | [production] | 
            
  | 14:25 | <zfilipin@tin> | Synchronized php-1.31.0-wmf.7/extensions/EventBus/EventBus.php: SWAT: [[gerrit:390211|Logging improvements]] [[gerrit:390212|Rename logged field to fix logstash mapping]] (duration: 00m 54s) | [production] | 
            
  | 14:09 | <urandom> | Decommissioning Cassandra, restbase2004-b.codfw.wmnet (T179422) | [production] | 
            
  | 13:04 | <moritzm> | rebooting mw1209-mw1220 (app servers) to 4.9.51 (also to pick up new OpenSSL) | [production] | 
            
  | 12:38 | <moritzm> | rebooting mw1189-mw1208 (API servers) to 4.9.5 (also to pick up new OpenSSL) | [production] | 
            
  | 12:37 | <Amir1> | ladsgroup@terbium:/srv/mediawiki-staging/php-1.31.0-wmf.6$ mwscript extensions/ORES/maintenance/CheckModelVersions.php --wiki=frwiki (T180115) | [production] | 
            
  | 11:51 | <moritzm> | rebooting mw1180-mw1188 (app servers) to 4.9.5 (also to pick up new OpenSSL) | [production] | 
            
  | 11:45 | <_joe_> | removed all local hacks from puppetmaster1001, now it uses rhodium again | [production] | 
            
  | 11:37 | <_joe_> | cleaning up spurious directories /var/lib/puppet/server/ssl/ca from eqiad's puppetmaster backends, generated due to some error on 8/11/2017 | [production] | 
            
  | 11:10 | <ema> | powercycle cp2008, stuck rebooting | [production] | 
            
  | 11:03 | <moritzm> | rebooting mw1276-mw1279 (API canaries) to 4.9.5 (also to pick up new OpenSSL) | [production] | 
            
  | 09:50 | <moritzm> | rolling reboot of scb in eqiad for kernel update (also to pick up openssl updates) | [production] | 
            
  | 07:56 | <ema> | cp1074 failed rebooting, power-cycled | [production] | 
            
  | 07:46 | <_joe_> | restarting apache on rhodium after setting --profile --trace in the puppet settings | [production] | 
            
  | 07:04 | <legoktm@tin> | Synchronized php-1.31.0-wmf.7/resources/: Restore jquery.badge and jquery.placeholder modules (duration: 00m 53s) | [production] | 
            
  | 02:58 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Nov  9 02:58:30 UTC 2017 (duration 6m 59s) | [production] | 
            
  | 02:51 | <l10nupdate@tin> | scap sync-l10n completed (1.31.0-wmf.7) (duration: 09m 09s) | [production] | 
            
  | 02:29 | <l10nupdate@tin> | scap sync-l10n completed (1.31.0-wmf.6) (duration: 09m 18s) | [production] | 
            
  | 01:02 | <twentyafterfour> | Not deploying any phabricator updates this week. | [production] | 
            
  
    | 2017-11-08
      
      § | 
    
  | 23:45 | <urandom> | Decommissioning Cassandra, restbase2004.codfw.wmnet (T179422) | [production] | 
            
  | 22:50 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.7 try #2 | [production] | 
            
  | 22:43 | <ebernhardson@tin> | Synchronized php-1.31.0-wmf.6/extensions/CirrusSearch/: Backport cirrus rescore profile refactor to wmf.6 (duration: 01m 02s) | [production] | 
            
  | 22:21 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 back to wmf.6 | [production] | 
            
  | 22:09 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.7 | [production] | 
            
  | 21:44 | <bsitzmann@tin> | Finished deploy [mobileapps/deploy@00e60b2]: Update mobileapps to 8e82983 (T178706 T178708 T178333 T170692) (duration: 07m 12s) | [production] | 
            
  | 21:37 | <bsitzmann@tin> | Started deploy [mobileapps/deploy@00e60b2]: Update mobileapps to 8e82983 (T178706 T178708 T178333 T170692) | [production] | 
            
  | 21:34 | <ejegg> | re-started donations queue consumer with new thank you letter | [production] | 
            
  | 21:09 | <ejegg> | updated CiviCRM from b11c59130e7f17425da3024067441a2635fffe55 to ddc388195e1974e17c1f6f3f67f285898c0eb2bb | [production] | 
            
  | 20:52 | <ejegg> | turned off donations consumer for ty letter update | [production] | 
            
  | 20:36 | <ejegg> | updated payments-wiki from a539d275377b251762f4935c3014978253f39c08 to d150287f0b64d13589e1af728be0cb5b89f66699 | [production] | 
            
  | 20:33 | <aaron@tin> | Synchronized php-1.31.0-wmf.6/extensions/CentralAuth: Use the proper cache key method in loadFromCache() (duration: 00m 54s) | [production] | 
            
  | 20:22 | <apergos> | rsync from ms1001 to labstore1006 of dumps, 17T so expect it to take several days | [production] | 
            
  | 20:10 | <herron> | depooled rhodium via puppetmaster1001 apache config | [production] | 
            
  | 20:09 | <demon@tin> | Synchronized php: symlink (duration: 00m 49s) | [production] | 
            
  | 19:57 | <demon@tin> | Synchronized php-1.31.0-wmf.7/extensions/CentralAuth/includes/CentralAuthUser.php: (no justification provided) (duration: 00m 51s) | [production] | 
            
  | 19:55 | <urandom> | Creating mathoid schema (T179419) | [production] | 
            
  | 19:31 | <niharika29@tin> | Synchronized php-1.31.0-wmf.6/extensions/ORES/: Store stats of accessing ores service for getting thresholds T179862 (duration: 00m 51s) | [production] | 
            
  | 19:31 | <urandom> | Creating page restrictions schema (T179421) | [production] | 
            
  | 19:30 | <niharika29@tin> | Synchronized php-1.31.0-wmf.7/extensions/ORES/: Store stats of accessing ores service for getting thresholds T179862 (duration: 00m 51s) | [production] |