| 2014-10-22
      
      § | 
    
  | 23:22 | <demon> | Synchronized php-1.25wmf4/extensions/Collection: (no message) (duration: 00m 04s) | [production] | 
            
  | 23:21 | <demon> | Synchronized php-1.25wmf3/extensions/Collection: (no message) (duration: 00m 04s) | [production] | 
            
  | 23:08 | <mutante> | added jhernandez to wmf LDAP group | [production] | 
            
  | 22:54 | <bd808> | forced puppet run on logstash1003 to pick up https://gerrit.wikimedia.org/r/#/c/168199/ | [production] | 
            
  | 22:47 | <bd808> | forced puppet run on logstash1002 to pick up https://gerrit.wikimedia.org/r/#/c/168199/ | [production] | 
            
  | 22:42 | <bd808> | forced puppet run on logstash1001 to pick up https://gerrit.wikimedia.org/r/#/c/168199/ | [production] | 
            
  | 22:21 | <bblack> | depooled amssq42 (esams text) for trusty testing | [production] | 
            
  | 21:38 | <bd808> | killed duplicate logstash services running on logstash1001 and restarted | [production] | 
            
  | 21:16 | <arlolra> | updated OCG to version e977e2c8ecacea2b4dee837933cc2ffdc6b214cb | [production] | 
            
  | 21:07 | <bd808> | forced puppet run on logstash1001 to pick up https://gerrit.wikimedia.org/r/#/c/168182/ | [production] | 
            
  | 20:40 | <bd808> | forced puppet run on logstash1001 to pick up https://gerrit.wikimedia.org/r/168089 | [production] | 
            
  | 20:37 | <mutante> | powercycling unresponsive ms-be1012 (this happened before, search SAL for hostname) | [production] | 
            
  | 20:25 | <arlolra> | updated Parsoid to version 2a8dc85ce676391acd8c6255c4f94250612c9ee2 | [production] | 
            
  | 16:59 | <andrewbogott> | reinstalling OS on virt1006 | [production] | 
            
  | 15:57 | <marktraceur> | Synchronized wmf-config/InitialiseSettings.php: [SWAT] Send collection logs to logstash. (duration: 00m 05s) | [production] | 
            
  | 15:31 | <ottomata> | setting request.required.acks to 2 for mobile and text varnishkafka's (mobile was set to 2 yesterday) | [production] | 
            
  | 15:24 | <_joe_> | rolling restart of hhvm appservers, to alleviate memory issues | [production] | 
            
  | 15:09 | <marktraceur> | Synchronized wmf-config/InitialiseSettings.php: [SWAT] Pre-render thumbnails on upload on Commons (duration: 00m 05s) | [production] | 
            
  | 15:05 | <marktraceur> | Synchronized wmf-config/CommonSettings-labs.php: [SWAT] Re-enable PediaPress POD in production. (duration: 00m 05s) | [production] | 
            
  | 15:05 | <marktraceur> | Synchronized wmf-config/CommonSettings.php: [SWAT] Re-enable PediaPress POD in production. (duration: 00m 05s) | [production] | 
            
  | 13:38 | <godog> | catch-up swiftrepl sync eqiad -> codfw for commons containers | [production] | 
            
  | 09:41 | <hashar> | Zuul/Jenkins in a deadlock | [production] | 
            
  | 09:08 | <hashar> | Restarting Jenkins | [production] | 
            
  | 09:07 | <hashar> | Jenkins: upgrading gearman-plugin from 0.0.7-1-g3811bb8 to 0.1.0-1-gfa5f083 .  Ie bring us to latest version + 1 commit | [production] | 
            
  | 03:54 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Oct 22 03:54:53 UTC 2014 (duration 54m 52s) | [production] | 
            
  | 02:30 | <LocalisationUpdate> | completed (1.25wmf4) at 2014-10-22 02:30:07+00:00 | [production] | 
            
  | 02:17 | <LocalisationUpdate> | completed (1.25wmf3) at 2014-10-22 02:17:49+00:00 | [production] | 
            
  
    | 2014-10-21
      
      § | 
    
  | 23:21 | <maxsem> | Synchronized php-1.25wmf4/includes/PrefixSearch.php: https://gerrit.wikimedia.org/r/#/c/167982/ (duration: 00m 03s) | [production] | 
            
  | 23:09 | <maxsem> | Synchronized php-1.25wmf3/extensions/MobileFrontend/: SWAT (duration: 00m 04s) | [production] | 
            
  | 23:08 | <maxsem> | Synchronized php-1.25wmf4/extensions/MobileFrontend/: SWAT (duration: 00m 04s) | [production] | 
            
  | 23:07 | <maxsem> | Synchronized php-1.25wmf4/extensions/CentralAuth/: SWAT (duration: 00m 04s) | [production] | 
            
  | 22:48 | <mutante> | radium - installed OS, signing puppet cert requests, initial run ... | [production] | 
            
  | 22:42 | <legoktm> | manually finished global rename for BonumTV --> Karypal which failed due to page move timeout | [production] | 
            
  | 22:13 | <andrewbogott> | deleted unused labs projects:  versionview, feeds, datadog, fundraising-awight, simplewiki, mediawiki-custom-de, fundraising, sartoris, wikibits, incubator, wikiversity-sandbox, data4all | [production] | 
            
  | 21:33 | <K4-7131> | adjusting payments antifraud filters | [production] | 
            
  | 21:29 | <ebernhardson> | Synchronized php-1.25wmf4/extensions/LiquidThreads/api/ApiQueryLQTThreads.php: Bump LQT in 1.25wmf4 (duration: 00m 04s) | [production] | 
            
  | 20:41 | <cscott> | updated OCG to version 523c8123cd826c75240837c42aff6301032d8ff1 | [production] | 
            
  | 20:40 | <hoo> | Synchronized wmf-config/CommonSettings.php: Set extendwatchlist = 0 (duration: 00m 08s) | [production] | 
            
  | 20:37 | <hashar> | lanthanum /var/lib/jenkins-slave/tmpfs went full again. cleared up a bunch of files | [production] | 
            
  | 20:22 | <_joe_> | installing new hhvm packages on the depooled server mw1189, for debugging | [production] | 
            
  | 20:14 | <AaronSchulz> | Deployed deployment/jobrunner to d426235e10edc682b532e7b4f2b02bb9414661ba | [production] | 
            
  | 19:47 | <hoo> | Synchronized wmf-config/InitialiseSettings.php: Add import sources for orwikisource (duration: 00m 08s) | [production] | 
            
  | 19:12 | <qchris> | Added its-phabricator plugin (d425a5ded909ee73df53d5e6d91d28014d0be375) into gerrit | [production] | 
            
  | 18:40 | <chasemp> | puppet on gerrit which restarted service | [production] | 
            
  | 18:25 | <reedy> | Synchronized php-1.25wmf3/extensions/Collection/: (no message) (duration: 00m 13s) | [production] | 
            
  | 18:24 | <reedy> | Synchronized php-1.25wmf4/extensions/Collection/: (no message) (duration: 00m 14s) | [production] | 
            
  | 18:21 | <reedy> | Synchronized database lists: (no message) (duration: 00m 26s) | [production] | 
            
  | 18:20 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 18s) | [production] | 
            
  | 18:08 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf4 | [production] | 
            
  | 16:54 | <_joe_> | stopping puppet on mw1114 in order to do some jemalloc debugging | [production] |