| 
      
        2017-01-27
      
      §
     | 
  
    
  | 21:36 | 
  <mobrovac@tin> | 
  Finished deploy [trending-edits/deploy@0e79bec]: Bump max_age to 12h T156411 (duration: 01m 58s) | 
  [production] | 
            
  | 21:34 | 
  <mobrovac@tin> | 
  Starting deploy [trending-edits/deploy@0e79bec]: Bump max_age to 12h T156411 | 
  [production] | 
            
  | 21:28 | 
  <mobrovac@tin> | 
  Finished deploy [trending-edits/deploy@e0e32bb]: Restart the service to assess the load of replaying the last 6h T156411 (duration: 01m 03s) | 
  [production] | 
            
  | 21:27 | 
  <mobrovac@tin> | 
  Starting deploy [trending-edits/deploy@e0e32bb]: Restart the service to assess the load of replaying the last 6h T156411 | 
  [production] | 
            
  | 20:24 | 
  <jynus> | 
  restart and upgrade mariadb on db1048 | 
  [production] | 
            
  | 20:14 | 
  <mutante> | 
  db1029, db1042, analytics1015, analytics1026 - puppet node deactivate, remove from icinga, finish decom (T147313, T149793, T146265) | 
  [production] | 
            
  | 20:07 | 
  <volans> | 
  updated watchmouse checks for s5 (de wiki) because Main_Page was deleted, used the localized page instead | 
  [production] | 
            
  | 19:52 | 
  <mutante> | 
  db1019 - shutdown -h now (T146265) | 
  [production] | 
            
  | 19:51 | 
  <mutante> | 
  db1042 - i came to shut it down .. and noticed it had died (or somebody did it) about 3 hours ago .. there it goes (T149793) | 
  [production] | 
            
  | 18:52 | 
  <twentyafterfour@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group2 wikis to 1.29.0-wmf.9 | 
  [production] | 
            
  | 18:52 | 
  <twentyafterfour> | 
  Rolling forward with group2 to 1.29.0-wmf.9 refs T156364 T154683 | 
  [production] | 
            
  | 17:55 | 
  <papaul> | 
  OS installation on mc2019-mc2036 | 
  [production] | 
            
  | 16:08 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: db1072 change IP - T156226 (duration: 00m 40s) | 
  [production] | 
            
  | 16:07 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: db1072 change IP - T156226 (duration: 00m 40s) | 
  [production] | 
            
  | 16:01 | 
  <jynus> | 
  submitted wmf-mariadb10_10.0.29-2 for T156373 fix | 
  [production] | 
            
  | 15:48 | 
  <marostegui> | 
  Stop mysql and shutdown db1072 for maintenance - T156226 | 
  [production] | 
            
  | 15:45 | 
  <ema> | 
  cache_text: ban req.url == "/apple-app-site-association" && obj.status == 404 (T155504) | 
  [production] | 
            
  | 14:13 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Add rack positions for s7 in codfw (duration: 00m 40s) | 
  [production] | 
            
  | 14:12 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Add rack positions for s7 in eqiad (duration: 00m 40s) | 
  [production] | 
            
  | 13:44 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Add rack positions for s6 in eqiad (duration: 00m 40s) | 
  [production] | 
            
  | 13:44 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Add rack positions for s6 in codfw (duration: 00m 40s) | 
  [production] | 
            
  | 13:31 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Add rack positions for s5 in eqiad (duration: 00m 40s) | 
  [production] | 
            
  | 13:30 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Add rack positions for s5 in codfw (duration: 00m 40s) | 
  [production] | 
            
  | 13:14 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Add rack positions for s4 in codfw (duration: 00m 41s) | 
  [production] | 
            
  | 13:13 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Add rack positions for s4 in eqiad (duration: 00m 43s) | 
  [production] | 
            
  | 13:11 | 
  <jynus> | 
  starting db1048 until db1043-bin.001457:753455353, expect it to stop soon | 
  [production] | 
            
  | 13:00 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Add rack positions for s3 in eqiad (duration: 00m 40s) | 
  [production] | 
            
  | 13:00 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Add rack positions for s3 in codfw (duration: 00m 40s) | 
  [production] | 
            
  | 12:24 | 
  <moritzm> | 
  upgrading mediawiki canaries to new openssl 1.1 package | 
  [production] | 
            
  | 12:13 | 
  <moritzm> | 
  upgrading openjdk-7 packages (security updates) on wdqs cluster | 
  [production] | 
            
  | 11:56 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Add rack positions for s2 in codfw (duration: 00m 47s) | 
  [production] | 
            
  | 11:55 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Add rack positions for s2 in eqiad (duration: 00m 59s) | 
  [production] | 
            
  | 11:11 | 
  <moritzm> | 
  initial installation of openssl bugfix/security updates | 
  [production] | 
            
  | 10:54 | 
  <moritzm> | 
  uploaded openssl 1.0.2k for jessie-wikimedia to carbon | 
  [production] | 
            
  | 10:35 | 
  <paravoid> | 
  manually running certspotter -all_time as my user on einstenium (will take a few days to complete) | 
  [production] | 
            
  | 10:21 | 
  <legoktm> | 
  added addshore to labs-tools-wikibugs2 gerrit group | 
  [production] | 
            
  | 08:13 | 
  <moritzm> | 
  uploaded openssl 1.1.0d packages for jessie-wikimedia to carbon | 
  [production] | 
            
  | 02:58 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Fri Jan 27 02:58:52 UTC 2017 (duration 5m 46s) | 
  [production] | 
            
  | 02:53 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.9) (duration: 14m 14s) | 
  [production] | 
            
  | 02:21 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.8) (duration: 07m 57s) | 
  [production] | 
            
  | 01:20 | 
  <twentyafterfour> | 
  deploying hotfix for phabricator refs T154479 | 
  [production] | 
            
  | 00:48 | 
  <mutante> | 
  carbon - moved the 1.5TB /srv/"mirrors.off", which used to be mirrors but is now on sodium, into /  to that /srv/ can be synced without this | 
  [production] | 
            
  | 00:32 | 
  <dereckson@tin> | 
  Synchronized wmf-config/throttle.php: Fix throttle rule for Her Girl Friday + Lenny Unconference (T156278) (duration: 00m 53s) | 
  [production] | 
            
  | 00:12 | 
  <volans> | 
  re-enabled puppet (with a temporary fix to keep parsoid-vd and parsoid-vd-client stopped) on ruthenium T156177 | 
  [production] |