| 2015-11-12
      
      § | 
    
  | 15:46 | <mobrovac> | restbase canary deploy on rb1001 of e5e45d11ad3f | [production] | 
            
  | 15:39 | <jynus> | powercycling mw1041 | [production] | 
            
  | 15:30 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1035 after maintenance (duration: 00m 29s) | [production] | 
            
  | 10:11 | <ori@tin> | Synchronized php-1.27.0-wmf.6/extensions/UniversalLanguageSelector: I9206a79: Only use jQuery.tipsy for undo popover (duration: 00m 31s) | [production] | 
            
  | 09:53 | <jynus> | rebooting/upgrading/restarting mariadb on db1035 after maintenance finished | [production] | 
            
  | 09:51 | <jynus> | uploading new mariadb package linked to openssl instead of yassl | [production] | 
            
  | 09:44 | <ori@tin> | Synchronized php-1.27.0-wmf.5/extensions/UniversalLanguageSelector: I9206a79: Only use jQuery.tipsy for undo popover (duration: 01m 15s) | [production] | 
            
  | 09:39 | <moritzm> | corrected apt sources on db2057, db2058, db2062, db2065, db2066, db2067, db2068, db2069 | [production] | 
            
  | 08:42 | <moritzm> | restarted salt-master on palladium | [production] | 
            
  | 07:35 | <_joe_> | restarting pybal on lvs1004 | [production] | 
            
  | 02:23 | <l10nupdate@tin> | Synchronized php-1.27.0-wmf.5/cache/l10n: l10nupdate for 1.27.0-wmf.5 (duration: 06m 56s) | [production] | 
            
  | 01:02 | <gwicke> | restbase: finished full deploy of restbase-deploy ce4abe784 | [production] | 
            
  | 00:50 | <gwicke> | restbase: starting full deploy of restbase-deploy ce4abe784 | [production] | 
            
  | 00:37 | <gwicke> | restbase: canary deploy of restbase-deploy ce4abe784 on restbase1001 | [production] | 
            
  | 00:32 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Fix typo in QuickSurveys config (duration: 00m 31s) | [production] | 
            
  
    | 2015-11-11
      
      § | 
    
  | 23:48 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.6 | [production] | 
            
  | 23:44 | <twentyafterfour@tin> | Synchronized php-1.27.0-wmf.6/includes/jobqueue/aggregator/: sync https://gerrit.wikimedia.org/r/#/c/252588/ (duration: 00m 32s) | [production] | 
            
  | 21:13 | <subbu> | finished deploying parsoid sha 7ca999c1 | [production] | 
            
  | 21:06 | <subbu> | synced new code + restarted parsoid on wtp1001 as canary; monitoring graphs for a little bit | [production] | 
            
  | 21:02 | <subbu> | starting parsoid deploy | [production] | 
            
  | 20:52 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.5 | [production] | 
            
  | 19:51 | <hoo> | Started rebuildItemsPerSite for wikidatawiki on mw1152. Feel free to kill, should it cause troubles. | [production] | 
            
  | 19:48 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.6 | [production] | 
            
  | 19:37 | <twentyafterfour> | deploying 1.27.0-wmf.6 to group1 | [production] | 
            
  | 19:09 | <YuviPanda> | nfs-exports on labstore1001 failed because of http failure of wikitech | [production] | 
            
  | 19:05 | <YuviPanda> | started nfs-exports service on labstore1001 | [production] | 
            
  | 18:18 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Repool db2067 after maintenance (duration: 00m 29s) | [production] | 
            
  | 16:46 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.5/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: Restore satisfaction schema and fix the performance issue that it had part II [[gerrit:252349]] (duration: 00m 30s) | [production] | 
            
  | 16:45 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.5/extensions/WikimediaEvents/WikimediaEvents.php: SWAT: Restore satisfaction schema and fix the performance issue that it had part I [[gerrit:252349]] (duration: 00m 30s) | [production] | 
            
  | 16:34 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.6/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: Restore satisfaction schema and fix the performance issue that it had part II [[gerrit:252347]] (duration: 00m 30s) | [production] | 
            
  | 16:33 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.6/extensions/WikimediaEvents/WikimediaEvents.php: SWAT: Restore satisfaction schema and fix the performance issue that it had part I [[gerrit:252347]] (duration: 00m 40s) | [production] | 
            
  | 16:17 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: First QuickSurvey for reader segmentation research - external survey [[gerrit:251133]] (duration: 00m 30s) | [production] | 
            
  | 16:02 | <mutante> | uploaded jenkins 1.625.2 to apt.wm.org, upgrading on gallium | [production] | 
            
  | 15:47 | <moritzm> | restarted kafka broker on kafka1022 | [production] | 
            
  | 15:34 | <moritzm> | stopping kafka broker on kafka1022 (to enable ferm) | [production] | 
            
  | 15:31 | <moritzm> | restarted kafka broker on kafka1020 | [production] | 
            
  | 15:25 | <moritzm> | stopping kafka1020 to enable ferm | [production] | 
            
  | 15:22 | <moritzm> | restarted kafka1018 to enable ferm | [production] | 
            
  | 15:19 | <moritzm> | stopping kafka1018 to enable ferm | [production] | 
            
  | 15:04 | <moritzm> | disabled puppet on kafka1018, kafka1020, kafka1022 (for enabling ferm) | [production] | 
            
  | 14:18 | <godog> | start cassandra instance a on restbase2001 | [production] | 
            
  | 13:54 | <mobrovac> | restbase deployed 0d961a2 (deploy end) | [production] | 
            
  | 13:42 | <mobrovac> | restbase deploying 0d961a2 | [production] | 
            
  | 11:28 | <godog> | reimage restbase2001.codfw.wmnet - take #2 | [production] | 
            
  | 10:16 | <jynus> | disabling puppet and restarting mysql on db2067 | [production] | 
            
  | 10:16 | <godog> | reimage restbase2001.codfw.wmnet | [production] | 
            
  | 10:07 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Depool db2067 for maintenance (not db1067) (duration: 00m 30s) | [production] | 
            
  | 10:05 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1067 for maintenance (duration: 00m 30s) | [production] | 
            
  | 09:57 | <twentyafterfour@tin> | Synchronized php-1.27.0-wmf.6/extensions/OAuth/: really fix T118372 this time (duration: 00m 29s) | [production] | 
            
  | 09:43 | <twentyafterfour@tin> | Synchronized php-1.27.0-wmf.6/extensions/OAuth/: oauth login broken by php error. This fixes T118372 (duration: 00m 31s) | [production] |