| 
      
        2016-07-08
      
      §
     | 
  
    
  | 20:51 | 
  <urandom> | 
  Throttle RESTBase Cassandra outgoing streams to 1mbit cluster-wide : T139362 (actually happened at 21:26) | 
  [production] | 
            
  | 20:17 | 
  <bd808> | 
  Deleted old l10nupdate caches manually on tin (T130317) | 
  [production] | 
            
  | 19:56 | 
  <urandom> | 
  Throttle RESTBase Cassandra outgoing streams to 3mbit cluster-wide : T139362 | 
  [production] | 
            
  | 18:52 | 
  <anomie> | 
  Attempting to resubmit LocalRenameUserJobs for T137973 | 
  [production] | 
            
  | 18:48 | 
  <urandom> | 
  "This is going to hurt me more than it does you."; `nodetool removenode' of restbase1014-c.eqiad.wmnet : T139362 | 
  [production] | 
            
  | 18:39 | 
  <urandom> | 
  Stopping restbase1014-c.eqiad.wmnet : T139362 | 
  [production] | 
            
  | 18:29 | 
  <mutante> | 
  db1042 - temp stop puppet, edit ferm rules to allow testing from lead | 
  [production] | 
            
  | 14:43 | 
  <jynus> | 
  rechecking data consistency after m3 table fixes (could cause lag) | 
  [production] | 
            
  | 12:21 | 
  <moritzm> | 
  installing glib updates from jessie point release | 
  [production] | 
            
  | 12:03 | 
  <jynus> | 
  stopping replication on db1043 (m3-slave) for maintenance | 
  [production] | 
            
  | 09:33 | 
  <elukey@palladium> | 
  conftool action : set/pooled=yes; selector: aqs1002.eqiad.wmnet | 
  [production] | 
            
  | 09:30 | 
  <elukey@palladium> | 
  conftool action : set/pooled=no; selector: aqs1002.eqiad.wmnet | 
  [production] | 
            
  | 09:30 | 
  <elukey> | 
  upgrading nodejs packages on aqs100[23] | 
  [production] | 
            
  | 09:10 | 
  <elukey@palladium> | 
  conftool action : set/pooled=yes:weight=5; selector: mw1261.eqiad.wmnet | 
  [production] | 
            
  | 09:06 | 
  <elukey@palladium> | 
  conftool action : set/pooled=no:weight=5; selector: mw1261.eqiad.wmnet | 
  [production] | 
            
  | 09:03 | 
  <elukey@palladium> | 
  conftool action : set/pooled=yes:weight=5; selector: mw1261.eqiad.wmnet | 
  [production] | 
            
  | 09:00 | 
  <moritzm> | 
  rebooting ruthenium for update to Linux 4.4 | 
  [production] | 
            
  | 08:40 | 
  <hashar> | 
  gallium: deleting old log files /var/log/zuul/gearman-server-debug.log* | 
  [production] | 
            
  | 08:35 | 
  <hashar> | 
  gallium: restarting Zuul to apply logging configuration change https://gerrit.wikimedia.org/r/#/c/291913/ | 
  [production] | 
            
  | 08:18 | 
  <moritzm> | 
  rebooting terbium for kernel security update | 
  [production] | 
            
  | 08:12 | 
  <moritzm> | 
  rearming keyholder on tin | 
  [production] | 
            
  | 08:08 | 
  <moritzm> | 
  rebooting tin for kernel security update | 
  [production] | 
            
  | 08:08 | 
  <oblivian@palladium> | 
  conftool action : set/pooled=no; selector: name=mw1261.eqiad.wmnet | 
  [production] | 
            
  | 07:15 | 
  <_joe_> | 
  removing 20 gb logfile from terbium, only useless debug info | 
  [production] | 
            
  | 02:37 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Fri Jul  8 02:37:29 UTC 2016 (duration 6m 22s) | 
  [production] | 
            
  | 02:31 | 
  <dereckson@tin> | 
  scap sync-l10n completed (1.28.0-wmf.9) (duration: 07m 47s) | 
  [production] | 
            
  | 02:12 | 
  <Dereckson> | 
  scap pull on terbium (was out of disk space during previous full scap) | 
  [production] | 
            
  | 02:00 | 
  <l10nupdate@tin> | 
  LocalisationUpdate failed: git pull of extensions failed | 
  [production] | 
            
  | 01:54 | 
  <mutante> | 
  terbium ran out of disk, deleted rotated nutcracker log | 
  [production] | 
            
  | 01:48 | 
  <dereckson@tin> | 
  Finished scap: Flow 297914, Echo 297919 297934, ORES 297916 (duration: 27m 48s) | 
  [production] | 
            
  | 01:26 | 
  <ostriches> | 
  gerrit: readded robots.txt to ytterbium for now | 
  [production] | 
            
  | 01:20 | 
  <dereckson@tin> | 
  Started scap: Flow 297914, Echo 297919 297934, ORES 297916 | 
  [production] | 
            
  | 00:42 | 
  <awight> | 
  roll back paymentswiki further, to 2fc573cbb94e833c4144aa9dad79de8ec374bb09 | 
  [production] | 
            
  | 00:39 | 
  <awight> | 
  roll back paymentswiki from f54ffb4fad0dc18079a813fbe25813dba36c64aa to c33ddfccf945bd075f0abff9e9de8c09f0174f89 | 
  [production] | 
            
  | 00:27 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.9/includes/api/ApiParse.php: API: Generate head items in the context of the given title (T139565) (duration: 00m 30s) | 
  [production] | 
            
  | 00:01 | 
  <awight> | 
  CentralNotice campaigns reenabled | 
  [production] | 
            
  | 00:00 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable ORES review tool as a beta feature in ptwiki (T139692) (duration: 00m 28s) | 
  [production] | 
            
  
    | 
      
        2016-07-07
      
      §
     | 
  
    
  | 23:57 | 
  <Dereckson> | 
  Ran extensions/ORES/maintenance/CheckModelVersions.php and extensions/ORES/maintenance/PopulateDatabase.php on ptwiki (T139692) | 
  [production] | 
            
  | 23:55 | 
  <awight> | 
  Update paymentswiki from c33ddfccf945bd075f0abff9e9de8c09f0174f89 to f54ffb4fad0dc18079a813fbe25813dba36c64aa | 
  [production] | 
            
  | 23:50 | 
  <Dereckson> | 
  Created table ores_classification on ptwiki from php-1.28.0-wmf.9/extensions/ORES/sql/ores_classification.sql | 
  [production] | 
            
  | 23:48 | 
  <Dereckson> | 
  Created table ores_model on ptwiki from php-1.28.0-wmf.9/extensions/ORES/sql/ores_model.sql | 
  [production] | 
            
  | 23:36 | 
  <dereckson@tin> | 
  Synchronized wmf-config/CommonSettings.php: Remove unused deprecated $wgStyleSheetPath ([[Gerrit:297511]]) (duration: 00m 27s) | 
  [production] | 
            
  | 23:23 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: VisualEditor: Move cite out of primary toolbar except on WP/WB/WV ([[Gerrit:296573]]) (duration: 00m 30s) | 
  [production] | 
            
  | 22:33 | 
  <maxsem@tin> | 
  Synchronized php-1.28.0-wmf.9/extensions/VisualEditor/: https://gerrit.wikimedia.org/r/#/c/297795/ and https://gerrit.wikimedia.org/r/#/c/297908/ (duration: 00m 45s) | 
  [production] | 
            
  | 21:32 | 
  <urandom> | 
  Bootstrapping restbase1009-c : T139362 | 
  [production] | 
            
  | 20:59 | 
  <urandom> | 
  Fin : T126629 | 
  [production] | 
            
  | 20:52 | 
  <urandom> | 
  Restarting Cassandra instance restbase2009-b.codfw.wmnet : T126629 | 
  [production] | 
            
  | 20:50 | 
  <urandom> | 
  Restarting Cassandra instance restbase2009-a.codfw.wmnet : T126629 | 
  [production] | 
            
  | 20:48 | 
  <urandom> | 
  Upgrading Cassandra to 2.2.6-wmf1 on restbase2009.codfw.wmnet : T126629 | 
  [production] | 
            
  | 20:48 | 
  <urandom> | 
  Cassandra upgrade of restbase2006.codfw.wmnet instances complete : T126629 | 
  [production] |