| 2017-07-20
      
      § | 
    
  | 06:24 | <mutante> | netmon1002 - librenms: fix permissions on /srv/librenms/rrd data after rsyncing, mismatching UIDs vs netmon1001 and rsyncd in chroot-issue | [production] | 
            
  | 06:00 | <oblivian@puppetmaster1001> | conftool action : set/pooled=inactive; selector: name=cp3048.esams.wmnet | [production] | 
            
  | 05:46 | <TimStarling> | on contint1001 restarted zuul and zuul-merger | [production] | 
            
  | 05:30 | <TimStarling> | on contint1001 restarted jenkins | [production] | 
            
  | 05:05 | <marostegui> | Configure replication for s2 on labsdb1009 and labsdb1010 - T153743 | [production] | 
            
  | 04:42 | <mutante> | netmon1002 - restarted Apache for LDAP issue - librenms.wm.org switched back to it, after rsyncing rrd data, re-enabling puppet | [production] | 
            
  | 04:05 | <andrewbogott> | restarting rabbitmq-server on labcontrol1001 | [production] | 
            
  | 03:34 | <andrewbogott> | service nova-network restart on labnet1001 | [production] | 
            
  | 03:32 | <andrewbogott> | service uwsgi-labspuppetbackend restart on labcontrol1001 | [production] | 
            
  | 03:02 | <l10nupdate@tin> | LocalisationUpdate failed: git pull of extensions failed | [production] | 
            
  | 02:22 | <mutante> | netmon1001 - rsyncing librenms rrd data to netmon1002 - T159756 | [production] | 
            
  | 01:17 | <andrewbogott> | restarting keystone on labcontrol1001 | [production] | 
            
  | 01:14 | <twentyafterfour> | phabricator upgrade complete | [production] | 
            
  | 01:10 | <twentyafterfour> | begin (belated) phabricator upgrade, expect momentary downtime. | [production] | 
            
  | 00:09 | <dereckson@tin> | Synchronized php-1.30.0-wmf.9/resources/src/mediawiki.widgets/mw.widgets.SearchInputWidget.js: Revert "Make mw.widgets.SearchInputWidget extend OO.ui.SearchInputWidget" (3/3) (duration: 00m 46s) | [production] | 
            
  | 00:08 | <dereckson@tin> | Synchronized php-1.30.0-wmf.9/resources/Resources.php: Revert "Make mw.widgets.SearchInputWidget extend OO.ui.SearchInputWidget" (2/3) (duration: 00m 46s) | [production] | 
            
  | 00:08 | <dereckson@tin> | Synchronized php-1.30.0-wmf.9/includes/widget/SearchInputWidget.php: Revert "Make mw.widgets.SearchInputWidget extend OO.ui.SearchInputWidget" (1/3) (duration: 00m 46s) | [production] | 
            
  | 00:06 | <dereckson@tin> | Synchronized php-1.30.0-wmf.10/resources/src/mediawiki.widgets/mw.widgets.SearchInputWidget.js: Revert "Make mw.widgets.SearchInputWidget extend OO.ui.SearchInputWidget" (3/3) (duration: 00m 46s) | [production] | 
            
  | 00:04 | <dereckson@tin> | Synchronized php-1.30.0-wmf.10/resources/Resources.php: Revert "Make mw.widgets.SearchInputWidget extend OO.ui.SearchInputWidget" (2/3) (duration: 00m 46s) | [production] | 
            
  | 00:03 | <dereckson@tin> | Synchronized php-1.30.0-wmf.10/includes/widget/SearchInputWidget.php: Revert "Make mw.widgets.SearchInputWidget extend OO.ui.SearchInputWidget" (1/3) (duration: 00m 46s) | [production] | 
            
  
    | 2017-07-19
      
      § | 
    
  | 23:29 | <dereckson@tin> | Synchronized wmf-config/Wikibase-production.php: Use correct class name for JsonUnitStorage (T171107) (duration: 00m 48s) | [production] | 
            
  | 22:14 | <reedy@tin> | Synchronized multiversion/: (no justification provided) (duration: 01m 11s) | [production] | 
            
  | 22:01 | <mutante> | hafnium, labmon1001 - restarted apache | [production] | 
            
  | 22:00 | <demon@tin> | Finished scap: all kinds of code style stuff for James_F & Reedy (duration: 05m 23s) | [production] | 
            
  | 21:59 | <mutante> | bromine _transparency.wm.org - restarted apache | [production] | 
            
  | 21:59 | <mutante> | dbmonitor2001 - restarted apache | [production] | 
            
  | 21:57 | <ejegg> | re-enabled CiviCRM de-dupe job | [production] | 
            
  | 21:56 | <mutante> | graphite200* - restarted apache | [production] | 
            
  | 21:54 | <demon@tin> | Started scap: all kinds of code style stuff for James_F & Reedy | [production] | 
            
  | 21:52 | <mutante> | netmon1003 - puppet run, restarted apache - fixed servermon.wikimedia.org | [production] | 
            
  | 21:50 | <mutante> | tegmen - restarted apache | [production] | 
            
  | 21:47 | <mutante> | netmon1001 - adding manual ferm rule for 80/443 - fixed librenms.wm.org | [production] | 
            
  | 21:44 | <mutante> | netmon1001 (librenms) - re-enable puppet once to get new CA, restart Apache, disable puppet again | [production] | 
            
  | 21:39 | <jynus> | reloading haproxy on dbproxy1005 for repooling db1009 | [production] | 
            
  | 21:35 | <mutante> | graphite1001 - restarted apache, ran puppet | [production] | 
            
  | 21:34 | <chasemp> | labstore1004/1005 puppet agent --test && service nslcd restart | [production] | 
            
  | 21:31 | <RainbowSprinkles> | running puppet & restarting gerrit/apache on cobalt/gerrit2001 | [production] | 
            
  | 21:29 | <mutante> | tungsten - restarted apache for CA change (xhgui) | [production] | 
            
  | 21:26 | <mutante> | logstash1001/1002 - restarted apache for CA change (logstash/kibana back) | [production] | 
            
  | 21:25 | <RainbowSprinkles> | Ran puppet and restarted apache on logstash100[1..3] | [production] | 
            
  | 21:23 | <madhuvishy> | Ran puppet and restarted apache on thorium (Runs hue, yarn, and pivot) | [production] | 
            
  | 21:21 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: most of group1 back on wmf.10 | [production] | 
            
  | 21:05 | <mutante> | krypton - run puppet, restart apache, fixed grafana-admin | [production] | 
            
  | 20:19 | <andrewbogott> | restaring slapd on seaborgium | [production] | 
            
  | 20:15 | <mobrovac@tin> | Finished deploy [restbase/deploy@3bb90c9]: (no justification provided) (duration: 09m 19s) | [production] | 
            
  | 20:13 | <chasemp> | seaborgium:~# service slapd restart | [production] | 
            
  | 20:12 | <chasemp> | serpens:~# service slapd restart | [production] | 
            
  | 20:05 | <mobrovac@tin> | Started deploy [restbase/deploy@3bb90c9]: (no justification provided) | [production] | 
            
  | 20:05 | <mobrovac@tin> | Finished deploy [restbase/deploy@3bb90c9] (staging): (no justification provided) (duration: 01m 39s) | [production] | 
            
  | 20:03 | <mobrovac@tin> | Started deploy [restbase/deploy@3bb90c9] (staging): (no justification provided) | [production] |