| 2015-02-12
      
      § | 
    
  | 16:24 | <krenair> | Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/187730/ (duration: 00m 05s) | [production] | 
            
  | 16:12 | <godog> | es-tool restart-fast on elastic1007 | [production] | 
            
  | 16:08 | <ottomata> | re-enabling bits varnishkafka instances | [production] | 
            
  | 16:01 | <krenair> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/190133/ (duration: 00m 07s) | [production] | 
            
  | 14:15 | <Krenair> | Manually logged a missing cross-wiki rights log change entry on meta "Avraham changed group membership for User:Bencmq@zhwiki from bureaucrat, check user and administrator to bureaucrat and administrator (requested)". See T89205 for details | [production] | 
            
  | 11:28 | <godog> | es-tool restart-fast on elastic1006 | [production] | 
            
  | 10:12 | <hashar> | gallium and lanthanum: dpkg --purge locate | [production] | 
            
  | 10:09 | <hashar> | gallium: uninstalling locate package from gallium. Has been installed on  2015-01-30  00:31:39 apparently manually by root@iron.wikimedia.org | [production] | 
            
  | 10:02 | <godog> | es-tool fast-restart on elastic1005 | [production] | 
            
  | 08:28 | <hashar> | puppet-lint now complains on error (not warnings) \\O/  {{bug:T87132}} | [production] | 
            
  | 04:54 | <springle> | broke puppet db grant. fixed puppet db grant | [production] | 
            
  | 04:53 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Feb 12 04:52:16 UTC 2015 (duration 52m 15s) | [production] | 
            
  | 04:09 | <springle> | sign puppet cert dbproxy1003, first run | [production] | 
            
  | 03:02 | <andrewbogott> | restarting wikitech-static.  shinken works! | [production] | 
            
  | 02:53 | <andrewbogott> | breaking wikitech-static on purpose to test the shinken alert | [production] | 
            
  | 02:42 | <LocalisationUpdate> | completed (1.25wmf17) at 2015-02-12 02:41:01+00:00 | [production] | 
            
  | 02:40 | <l10nupdate> | Synchronized php-1.25wmf17/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 02:24 | <LocalisationUpdate> | completed (1.25wmf16) at 2015-02-12 02:23:30+00:00 | [production] | 
            
  | 02:23 | <l10nupdate> | Synchronized php-1.25wmf16/cache/l10n: (no message) (duration: 00m 01s) | [production] | 
            
  | 00:33 | <catrope> | Synchronized php-1.25wmf17/resources/lib/oojs-ui: SWAT (duration: 00m 08s) | [production] | 
            
  | 00:15 | <catrope> | Synchronized php-1.25wmf16/resources/lib/oojs-ui: SWAT (duration: 00m 06s) | [production] | 
            
  | 00:13 | <aaron> | Synchronized wmf-config/StartProfiler.php: Use ProfilerSectionOnly to handle DB/filebackend entries (duration: 00m 05s) | [production] | 
            
  
    | 2015-02-11
      
      § | 
    
  | 23:02 | <twentyafterfour> | Purged l10n cache for 1.25wmf15 | [production] | 
            
  | 23:00 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf17 | [production] | 
            
  | 22:57 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.25wmf16 | [production] | 
            
  | 22:47 | <twentyafterfour> | Finished scap: testwiki to php-1.25wmf17 and rebuild l10n cache (duration: 48m 29s) | [production] | 
            
  | 22:45 | <mutante> | apt-get upgrading zirconium | [production] | 
            
  | 22:12 | <mutante> | deactivated ocg1003 in pybal | [production] | 
            
  | 22:05 | <andrewbogott> | updated wikitech-static to wmf/1.25wmf15 | [production] | 
            
  | 21:58 | <twentyafterfour> | Started scap: testwiki to php-1.25wmf17 and rebuild l10n cache | [production] | 
            
  | 21:38 | <subbu> | deployed parsoid version 4fc3b43d | [production] | 
            
  | 20:09 | <bblack> | eqiad-upload-https -> back to even weighting | [production] | 
            
  | 19:46 | <bblack> | all eqiad-upload-https -> cp1064 | [production] | 
            
  | 19:36 | <subbu> | temporarily turn off logging to logstash till logstash isssues are resolved. | [production] | 
            
  | 19:34 | <bblack> | repooled cp1070 (eqiad bits) in pybal | [production] | 
            
  | 19:18 | <bd808> | restarted elasticsearch on logstash1002 after OOM | [production] | 
            
  | 19:15 | <mutante> | moved docroots on zirconium to new logical volume for /srv | [production] | 
            
  | 19:09 | <godog> | powerdown graphite1002 T88992 | [production] | 
            
  | 18:20 | <bd808> | restarted Elasticsearch on logstash1003; preventative, other nodes restarted today | [production] | 
            
  | 17:59 | <mutante> | ran puppet on virt1000 - finished just fine, not sure why icinga said fail | [production] | 
            
  | 17:54 | <mutante> | running puppet on ruthenium (last was 2 days ago but also not admin disabled..) | [production] | 
            
  | 17:49 | <godog> | logging test | [production] | 
            
  | 10:18 | <godog> | restart elasticsearch on logstash1003, OOM | [production] | 
            
  | 09:58 | <hashar> | restarting Jenkins to upgrade the Credentials plugin | [production] | 
            
  | 06:29 | <springle> | Synchronized wmf-config/db-eqiad.php: repool db1057 (duration: 00m 05s) | [production] | 
            
  | 04:54 | <springle> | restarting labsdb1002 https://lists.wikimedia.org/pipermail/labs-l/2015-February/003354.html | [production] | 
            
  | 04:35 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Feb 11 04:34:30 UTC 2015 (duration 34m 29s) | [production] | 
            
  | 03:21 | <springle> | restarting labsdb1001 https://lists.wikimedia.org/pipermail/labs-l/2015-February/003354.html | [production] | 
            
  | 02:48 | <hoo> | Manually logged a missing global rights log change entry on meta "Ajraddatz changed global group membership for Benoit Rochon from (none) to OTRS-member with the following comment: [[Special:Diff/11227486|request]]". See also T89205 | [production] | 
            
  | 02:28 | <LocalisationUpdate> | completed (1.25wmf16) at 2015-02-11 02:26:55+00:00 | [production] |