| 
      
        2018-02-01
      
      §
     | 
  
    
  | 23:37 | 
  <mutante> | 
  creating new 100GB virtual disk for ganeti VM meitnerium (T186020) | 
  [production] | 
            
  | 23:12 | 
  <eileen> | 
  update civicrm revision changed from 849bba4186 to 71b1e35b99 (deploy civitoken) | 
  [production] | 
            
  | 22:37 | 
  <ejegg> | 
  updated payments-wiki from 40145892e7 to 341cb573a1 | 
  [production] | 
            
  | 21:56 | 
  <raita> | 
  Removed 2FA from User:Jehochman | 
  [production] | 
            
  | 21:52 | 
  <raita> | 
  Removed 2FA from User:Superzerocool (on Mon, Jan 29): https://phabricator.wikimedia.org/T185731 | 
  [production] | 
            
  | 20:50 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool of db1083 (duration: 00m 55s) | 
  [production] | 
            
  | 20:41 | 
  <jynus> | 
  deployed modified query killer to enwiki replicas | 
  [production] | 
            
  | 20:35 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: emergency depool of db1083 (duration: 00m 55s) | 
  [production] | 
            
  | 19:19 | 
  <chasemp> | 
  labservices1001:~# logrotate --force /etc/logrotate.conf | 
  [production] | 
            
  | 19:17 | 
  <chasemp> | 
  labservices1002:~# logrotate --force /etc/logrotate.conf | 
  [production] | 
            
  | 19:04 | 
  <demon@tin> | 
  Pruned MediaWiki: 1.31.0-wmf.16 [keeping static files] (duration: 01m 16s) | 
  [production] | 
            
  | 19:02 | 
  <demon@tin> | 
  Pruned MediaWiki: 1.31.0-wmf.15 (duration: 14m 55s) | 
  [production] | 
            
  | 16:26 | 
  <andrewbogott> | 
  apt-get install 'designate' on labservices1001 and 1002 — routine upgrade | 
  [production] | 
            
  | 15:39 | 
  <moritzm> | 
  upgrading nginx on mw1266-mw1299 (for T164456) | 
  [production] | 
            
  | 15:27 | 
  <moritzm> | 
  restarting apache/HHVM on deployment servers to pick up libxml2/curl security updates | 
  [production] | 
            
  | 15:14 | 
  <moritzm> | 
  installing curl security updates | 
  [production] | 
            
  | 14:48 | 
  <moritzm> | 
  installing tiff security updates | 
  [production] | 
            
  | 14:44 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool es2019 (duration: 00m 57s) | 
  [production] | 
            
  | 14:40 | 
  <moritzm> | 
  restarting nginx on sodium to pick up libxml2 security update | 
  [production] | 
            
  | 14:34 | 
  <moritzm> | 
  restarting apache on rutherfordium to pick up libxml2 security update | 
  [production] | 
            
  | 14:01 | 
  <moritzm> | 
  restarting nginx on puppetdb hosts to pick up libxml2 security update | 
  [production] | 
            
  | 13:56 | 
  <moritzm> | 
  restarting nginx on meitnerium/archiva to pick up libxml2 security update | 
  [production] | 
            
  | 13:42 | 
  <gehel> | 
  restarting nginx on wdqs* for upgrade | 
  [production] | 
            
  | 13:31 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Update db1051 reason for depooling (duration: 00m 56s) | 
  [production] | 
            
  | 13:23 | 
  <akosiaris> | 
  force puppet run on all postgres servers for https://gerrit.wikimedia.org/r/407424 | 
  [production] | 
            
  | 13:20 | 
  <jynus> | 
  stop and reimage es2019 | 
  [production] | 
            
  | 13:13 | 
  <moritzm> | 
  restarting apache on krypton to pick up libxml2 security update | 
  [production] | 
            
  | 13:13 | 
  <gehel> | 
  restarting postgresql and nodejs services on maps* | 
  [production] | 
            
  | 13:09 | 
  <gehel> | 
  upgrade nging on elastic* | 
  [production] | 
            
  | 12:54 | 
  <moritzm> | 
  restarting nginx on debug proxies to pick up libxml2 security update | 
  [production] | 
            
  | 12:53 | 
  <moritzm> | 
  restarting apache on hafnium to pick up libxml2 security update | 
  [production] | 
            
  | 12:06 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool es2018, depool es2019 (duration: 00m 57s) | 
  [production] | 
            
  | 12:03 | 
  <moritzm> | 
  restarting squid on URL downloaders to pick up libxml2 security update | 
  [production] | 
            
  | 11:53 | 
  <moritzm> | 
  installing libxml2 security updates | 
  [production] | 
            
  | 10:33 | 
  <godog> | 
  roll restart thumbor to lower subprocess timeout - T185479 | 
  [production] | 
            
  | 02:23 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.31.0-wmf.17) (duration: 05m 53s) | 
  [production] | 
            
  
    | 
      
        2018-01-31
      
      §
     | 
  
    
  | 23:56 | 
  <mutante> | 
  restarting apache on phabricator server, same pattern as described in T182832 | 
  [production] | 
            
  | 23:06 | 
  <bblack> | 
  re-pooling ulsfo in DNS - T185228 | 
  [production] | 
            
  | 23:04 | 
  <bblack> | 
  re-pooling ulsfo in DNS | 
  [production] | 
            
  | 23:00 | 
  <bblack> | 
  restarting ulsfo varnish-fe processes | 
  [production] | 
            
  | 22:55 | 
  <bblack> | 
  un-downtiming various ulsfo things | 
  [production] | 
            
  | 22:28 | 
  <mepps> | 
  updated civicrm from c70f01cd83 to 849bba4186 | 
  [production] | 
            
  | 22:04 | 
  <mepps> | 
  updated civicrm from c70f01cd83 to 63c918837c | 
  [production] | 
            
  | 21:57 | 
  <mholloway-shell@tin> | 
  Finished deploy [mobileapps/deploy@18d263a]: Update mobileapps to 3d717fa (duration: 06m 11s) | 
  [production] | 
            
  | 21:51 | 
  <mholloway-shell@tin> | 
  Started deploy [mobileapps/deploy@18d263a]: Update mobileapps to 3d717fa | 
  [production] | 
            
  | 21:35 | 
  <mutante> | 
  fixed icinga config for cp4024 parents | 
  [production] | 
            
  | 20:29 | 
  <demon@tin> | 
  Synchronized .gitmodules: consistency (duration: 00m 54s) | 
  [production] | 
            
  | 20:25 | 
  <demon@tin> | 
  Synchronized docroot/wikimedia.org/: bye bye firefox os. you will (not) be missed (duration: 00m 58s) | 
  [production] | 
            
  | 18:54 | 
  <robh@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=cp4032.ulsfo.wmnet | 
  [production] | 
            
  | 18:54 | 
  <robh@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=cp4031.ulsfo.wmnet | 
  [production] |