| 2016-02-03
      
      ยง | 
    
  | 22:34 | <moritzm> | repooling restbase1006 , depooling restbase1009 for kernel/Java update | [production] | 
            
  | 22:34 | <hashar> | Still looking at test.wikipedia.org being super "slow"  . scap still rebuilding though | [production] | 
            
  | 22:21 | <moritzm> | repooling restbase1005 , depooling restbase1006 for kernel/Java update | [production] | 
            
  | 22:14 | <hashar> | https://test.wikipedia.org/  switched to 1.27.0-wmf.12 | [production] | 
            
  | 22:13 | <hashar@mira> | Started scap: to properly sync other master tin due to l10nupdate ui mismatch | [production] | 
            
  | 22:13 | <subbu> | restarted parsoid on wtp1002 as a canary | [production] | 
            
  | 22:13 | <hashar@mira> | Finished scap: testwiki to php-1.27.0-wmf.12 and rebuild l10n cache (with proper branches for special_extensions) (duration: 20m 23s) | [production] | 
            
  | 22:07 | <moritzm> | repooling restbase1004 , depooling restbase1005 for kernel/Java update | [production] | 
            
  | 22:06 | <subbu> | starting parsoid deploy | [production] | 
            
  | 22:03 | <mutante> | mira, tin: find /srv/mediawiki-staging/ -uid 1001 -exec chown 10002 {} \; | [production] | 
            
  | 21:53 | <hashar> | reopened https://phabricator.wikimedia.org/T119165 ''l10nupdate user uid mismatch between tin and mira'' | [production] | 
            
  | 21:52 | <hashar@mira> | Started scap: testwiki to php-1.27.0-wmf.12 and rebuild l10n cache (with proper branches for special_extensions) | [production] | 
            
  | 21:51 | <mutante> | tin - find / -uid 1001 -exec chown 10002 {} \; | [production] | 
            
  | 21:49 | <mutante> | tin - fixing UID of l10nupdate user (T119165) | [production] | 
            
  | 21:45 | <moritzm> | depooling restbase1004 for kernel/Java update | [production] | 
            
  | 21:45 | <moritzm> | repooling restbase1003 | [production] | 
            
  | 21:35 | <hashar> | mismatching uid for l10nupdate user between mira and tin | [production] | 
            
  | 21:34 | <hashar@mira> | scap aborted: testwiki to php-1.27.0-wmf.12 and rebuild l10n cache (with proper branches for special_extensions) (duration: 07m 41s) | [production] | 
            
  | 21:32 | <moritzm> | depooling restbase1003 for kernel/Java update | [production] | 
            
  | 21:27 | <moritzm> | repooling restbase1008 | [production] | 
            
  | 21:26 | <hashar@mira> | Started scap: testwiki to php-1.27.0-wmf.12 and rebuild l10n cache (with proper branches for special_extensions) | [production] | 
            
  | 21:25 | <hashar> | mira had to hard reset  CentralNotice / SemanticMediaWiki / SemanticResultFormats / Validator after we pointed them from master to their proper branch, submodule attempted a rebase automatically.. That is a no no | [production] | 
            
  | 21:14 | <moritzm> | depooling restbase1008 for kernel/Java update | [production] | 
            
  | 21:08 | <hashar> | waiting for the submodule patch https://gerrit.wikimedia.org/r/#/c/268214/ to land and will scap again | [production] | 
            
  | 20:33 | <hashar@mira> | scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="testwiki" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.bTBpxD6CuI" ' returned non-zero exit status 1 (duration: 01m 13s) | [production] | 
            
  | 20:32 | <hashar@mira> | Started scap: testwiki to php-1.27.0-wmf.12 and rebuild l10n cache (after RandomRootPage had a dummy entry point added) | [production] | 
            
  | 20:31 | <demon@mira> | Synchronized php-1.27.0-wmf.12/extensions/RandomRootPage/: unbreak (duration: 01m 19s) | [production] | 
            
  | 20:23 | <hashar@mira> | scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_2188303825" --threads=10 --lang en  --quiet' returned non-zero exit status 255 (duration: 01m 49s) | [production] | 
            
  | 20:21 | <hashar@mira> | Started scap: testwiki to php-1.27.0-wmf.12 and rebuild l10n cache | [production] | 
            
  | 20:20 | <hashar> | Hacked wikiversions.json to only have testwiki on .12 | [production] | 
            
  | 19:58 | <demon@mira> | Synchronized wmf-config/InitialiseSettings.php: touch (duration: 01m 19s) | [production] | 
            
  | 19:49 | <demon@mira> | Synchronized wmf-config/: fix wikibase/mobilefrontend config (duration: 01m 19s) | [production] | 
            
  | 19:48 | <robh> | halting puppet on carbon for a few minutes to livehack a partition recipe change in netboot.cfg | [production] | 
            
  | 19:45 | <hashar> | https://phabricator.wikimedia.org/T125672  blocking wmf.12 "Notice: Undefined variable: wgMFQueryPropModules in /srv/mediawiki/wmf-config/Wikibase.php on line 120" | [production] | 
            
  | 19:39 | <akosiaris> | hot patch OTRS installation with https://github.com/OTRS/otrs/commit/c7ea6d64e02518e166fbac02f42f25dacad54342 | [production] | 
            
  | 19:35 | <hashar> | mira: manually fixed /php and /w/static/current symlinks to point back to .10  (wikiversions migrated them to .11 which we skip) | [production] | 
            
  | 19:30 | <moritzm> | repooling restbase1002 | [production] | 
            
  | 19:29 | <hashar> | Create patches to update wikiversions.json | [production] | 
            
  | 19:24 | <hashar> | Applying security patches on mira | [production] | 
            
  | 19:24 | <hashar> | starting train deployment of 1.27.0-wmf.12 | [production] | 
            
  | 19:09 | <csteipp> | deployed patch for T125684 | [production] | 
            
  | 19:08 | <moritzm> | depooling restbase1002 for kernel/Java update | [production] | 
            
  | 18:38 | <bd808@mira> | Synchronized wmf-config/InitialiseSettings-labs.php: Experiment one: Labs stripping HTML in beta (360e5af) (duration: 01m 19s) | [production] | 
            
  | 18:34 | <moritzm> | rebooting californium for kernel update | [production] | 
            
  | 18:16 | <bblack> | restarting pybal on lvs1001 | [production] | 
            
  | 18:04 | <jynus> | previous announcement was for db2011, not db2010 | [production] | 
            
  | 18:02 | <jynus> | starting slave IO thread on db2010 | [production] | 
            
  | 17:32 | <jynus@mira> | Synchronized wmf-config/db-eqiad.php: Repool db1060 after maintenance (duration: 01m 20s) | [production] | 
            
  | 17:22 | <mobrovac> | restbase restarting rb1001 | [production] | 
            
  | 17:03 | <mdholloway> | mobileapps deployed 68e38ec | [production] |