| 
      
        2014-01-31
      
      §
     | 
  
    
  | 05:03 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'depool db1052 for schema changes' | 
  [production] | 
            
  | 04:57 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'repool db1020, warm up' | 
  [production] | 
            
  | 03:41 | 
  <tstarling> | 
  finished scap: mostly no-op, did a reset to revert the wikidata changes (duration: 05m 00s) | 
  [production] | 
            
  | 03:37 | 
  <tstarling> | 
  started scap: mostly no-op, did a reset to revert the wikidata changes | 
  [production] | 
            
  | 02:07 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at 2014-01-31 02:07:39+00:00 | 
  [production] | 
            
  | 02:02 | 
  <LocalisationUpdate> | 
  completed (1.23wmf12) at 2014-01-31 02:02:37+00:00 | 
  [production] | 
            
  | 02:02 | 
  <LocalisationUpdate> | 
  completed (1.23wmf11) at 2014-01-31 02:01:50+00:00 | 
  [production] | 
            
  | 01:25 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'depool db1050 for schema changes' | 
  [production] | 
            
  | 00:19 | 
  <springle> | 
  xtrabackup clone db1042 to db1020 | 
  [production] | 
            
  | 00:19 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at 2014-01-31 00:19:28+00:00 | 
  [production] | 
            
  | 00:10 | 
  <LocalisationUpdate> | 
  completed (1.23wmf12) at 2014-01-31 00:10:17+00:00 | 
  [production] | 
            
  
    | 
      
        2014-01-30
      
      §
     | 
  
    
  | 23:40 | 
  <LocalisationUpdate> | 
  completed (1.23wmf11) at 2014-01-30 23:40:27+00:00 | 
  [production] | 
            
  | 23:14 | 
  <demon> | 
  synchronized wmf-config/Wikibase.php  'I50dfdc42: Enable quantity values' | 
  [production] | 
            
  | 22:59 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikidatawiki to wmf12 | 
  [production] | 
            
  | 22:36 | 
  <RobH> | 
  powercycling mw1061, system frozen | 
  [production] | 
            
  | 22:35 | 
  <RobH> | 
  rebooting frozen systems mw1057, mw1058, mw1060 | 
  [production] | 
            
  | 22:22 | 
  <RobH> | 
  powercycleing mw1039 as its also crashed | 
  [production] | 
            
  | 22:19 | 
  <RobH> | 
  mw1033 crashed, powercycling | 
  [production] | 
            
  | 22:14 | 
  <RobH> | 
  mw1036 crashed, unresponsive to console or ssh, rebooting | 
  [production] | 
            
  | 21:09 | 
  <reedy> | 
  synchronized php-1.23wmf12/extensions/Wikidata/ | 
  [production] | 
            
  | 20:38 | 
  <reedy> | 
  finished scap: Scap take 2 for 1.23wmf12 (duration: 18m 08s) | 
  [production] | 
            
  | 20:23 | 
  <reedy> | 
  started scap: Scap take 2 for 1.23wmf12 | 
  [production] | 
            
  | 20:16 | 
  <reedy> | 
  synchronized php-1.23wmf12/extensions/ContactPageFundraiser | 
  [production] | 
            
  | 20:04 | 
  <reedy> | 
  synchronized php-1.23wmf12/extensions/ContactPageFundraiser/ContactPage.php | 
  [production] | 
            
  | 19:55 | 
  <Reedy> | 
  Update indexes on wb_terms for testwikidatawiki https://gerrit.wikimedia.org/r/#/c/99660 | 
  [production] | 
            
  | 19:53 | 
  <Reedy> | 
  Changed wb_items_per_site.ips_row_id and wb_terms.term_row_id to BIGINT on testwikidatawiki | 
  [production] | 
            
  | 19:43 | 
  <reedy> | 
  synchronized php-1.23wmf12/includes/api/ApiQueryRevisions.php  'bug 60635' | 
  [production] | 
            
  | 19:36 | 
  <reedy> | 
  synchronized php-1.23wmf12/extensions/PdfHandler | 
  [production] | 
            
  | 19:31 | 
  <manybubbles> | 
  rebuilding search index on test2wiki went perfectly.  proceeding with test, testwikidatawiki, and mediawikiwiki | 
  [production] | 
            
  | 19:29 | 
  <manybubbles> | 
  rebuilding search index for test2wiki and checking that everything is sane | 
  [production] | 
            
  | 19:25 | 
  <awight_> | 
  updated crm from 2516fe6fe95c1915c981fcad287fdd6119202344 to 600e09e35c59c3b8c127e5f3c3cf2df11db4f20e | 
  [production] | 
            
  | 19:20 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 19:20 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: group0 wikis to 1.23wmf12 | 
  [production] | 
            
  | 19:19 | 
  <reedy> | 
  updated /a/common to {{Gerrit|Ie5c11239f}}: Update php symlink to php-1.23wmf11 | 
  [production] | 
            
  | 19:08 | 
  <Reedy> | 
  testwiki back to 1.23wmf11 even | 
  [production] | 
            
  | 19:06 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: all wikipedias to 1.23wmf11. testwiki back to 1.23wmf10 | 
  [production] | 
            
  | 19:02 | 
  <reedy> | 
  finished scap: testwiki to 1.23wmf12 and build l10n cache (duration: 18m 45s) | 
  [production] | 
            
  | 19:01 | 
  <awight_> | 
  crm updated from 9dc1887ce8391ca3a97dbb52d1d23454bbb21b03 to 2516fe6fe95c1915c981fcad287fdd6119202344 | 
  [production] | 
            
  | 18:57 | 
  <^d> | 
  jenkins: aborted another one of those pywiki jobs, was starting to back things up again. this job is broken methinks. | 
  [production] | 
            
  | 18:46 | 
  <reedy> | 
  started scap: testwiki to 1.23wmf12 and build l10n cache | 
  [production] | 
            
  | 18:42 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 18:42 | 
  <reedy> | 
  synchronized php-1.23wmf12  'staging' | 
  [production] | 
            
  | 18:32 | 
  <^d> | 
  jenkins: killed job, queue seems to be trying to catch up now | 
  [production] | 
            
  | 18:31 | 
  <^d> | 
  jenkins backed up due to pywikibot-core-test job stuck again | 
  [production] | 
            
  | 18:26 | 
  <awight_> | 
  crm updated from 7c675028cf9a51d4b2d6c8f0d716c343225eb0db to 9dc1887ce8391ca3a97dbb52d1d23454bbb21b03 | 
  [production] | 
            
  | 17:51 | 
  <reedy> | 
  updated /a/common to {{Gerrit|Id0633e5a0}}: depool db1020 for schema changes | 
  [production] | 
            
  | 17:41 | 
  <cmjohnson1_> | 
  labnet1001 down for card install | 
  [production] | 
            
  | 17:26 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Id0dea6e4: Revert "New extra language for wikidata: Ottoman Turkish (ota)"' | 
  [production] | 
            
  | 17:20 | 
  <awight_> | 
  crm updated from 704a029a01e2cbe0e99625f78cd472edbd86cd64 to 7c675028cf9a51d4b2d6c8f0d716c343225eb0db | 
  [production] | 
            
  | 04:09 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'depool db1020 for schema changes' | 
  [production] |