|
2012-05-14
§
|
| 18:26 |
<reedy> |
synchronized php-1.20wmf3 'Initial pushing of php-1.20wmf3 files to apaches' |
[production] |
| 18:12 |
<Reedy> |
Killing old php-1.20wmf1 directories from apaches to save full disks |
[production] |
| 13:48 |
<mutante> |
copying outdated wikiversions.dat/.cdb files from /home to /usr/local on spence, which fixes check_job_queue (thanks jeremyb) |
[production] |
| 13:07 |
<mutante> |
opening a bz bug for check_job_queue issue related to CommonSettings.php [[BZ:36835]] |
[production] |
| 07:43 |
<mutante> |
still upgrading/rebooting a couple srv (API) application servers with long uptime |
[production] |
| 06:22 |
<apergos> |
restarted lucene search on search1016 it had stopped doing anything useful (see ganglia graphs, also nothinig wtitten to logs) |
[production] |
| 02:22 |
<LocalisationUpdate> |
completed (1.20wmf2) at Mon May 14 02:22:09 UTC 2012 |
[production] |
|
2012-05-11
§
|
| 22:10 |
<preilly> |
synchronized wmf-config 'add wikimedia to zero image disable list fix header' |
[production] |
| 21:52 |
<preilly> |
synchronized wmf-config 'add wikimedia to zero image disable list' |
[production] |
| 19:49 |
<Reedy> |
ran apache-graceful-all |
[production] |
| 19:42 |
<RobH> |
apache restarted by puppet run on srv286 |
[production] |
| 19:31 |
<RobH> |
shutting down srv286 and srv286 for power rebalancing |
[production] |
| 19:23 |
<RobH> |
srv260 and srv261 back in business |
[production] |
| 19:10 |
<RobH> |
srv261 & srv261 shutting down for power rebalancing within the rack |
[production] |
| 18:33 |
<notpeter> |
shutting down search 13-20 for hd upgrades |
[production] |
| 18:05 |
<maplebed> |
swift: deleting the unsharded version of all sharded containers |
[production] |
| 18:03 |
<catrope> |
synchronized php-1.20wmf2/extensions/UploadWizard/ 'Deploy 4b5df1a1151ac80e309d396102e5e2a8d0c27ccb' |
[production] |
| 17:46 |
<maplebed> |
deleted wikipedia-de-local-thumb container from swift. the sharded version is currently being used. |
[production] |
| 15:33 |
<mutante> |
adding DNS entries for analytics hosts in new vlan 1121 (10.64.21.0/24), hosts starting at .101 to match names analytics1001 = .101 and ++ |
[production] |
| 15:03 |
<mutante> |
mw62 -unless somebody was on that right now it died. mgmt also just Create Instance Error |
[production] |
| 14:06 |
<mutante> |
kernel upgrading / rebooting srv servers where uptime > 200 d order by uptime desc limit 1 |
[production] |
| 13:12 |
<mutante> |
installing package upgrades on pdf1-3 (and installed requested indic fonts via new puppet role class) |
[production] |
| 11:39 |
<mutante> |
starting ms-be swift-container-auditors every once in a while |
[production] |
| 11:35 |
<mutante> |
stat1 - installed new kernel, but waiting to reboot. schedule with aotto |
[production] |
| 11:24 |
<mutante> |
upgrading packages/kernel on hooper, rebooting (Blog,Etherpad,Racktables) |
[production] |
| 09:21 |
<mutante> |
ekrem was close running out of disk again. logrotated apache logs, changed config to: size 512M,rotate 3 |
[production] |
| 08:58 |
<mutante> |
package upgrades on ekrem (IRC server, WAP, Apple dict...) |
[production] |
| 08:51 |
<mutante> |
rebooting marmontel (blog) |
[production] |
| 08:49 |
<Guest42776> |
upgrading apache/mysql/kernel on marmontel (blog) |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.20wmf2) at Fri May 11 02:20:39 UTC 2012 |
[production] |
| 02:00 |
<RoanKattouw> |
Started Apache back up on srv200, done debugging |
[production] |
| 01:58 |
<catrope> |
synchronized php-1.20wmf2/extensions/UserDailyContribs/UserDailyContribs.hooks.php 'Deploy 3c45831ffe1817f3dc18f06644db46b1b74173e7' |
[production] |
| 01:17 |
<RoanKattouw> |
Stopping Apache on srv200 so I can use it as my guinea pig for segfault debugging |
[production] |
| 00:56 |
<tstarling> |
synchronized php-1.20wmf2/includes/User.php 'header log' |
[production] |
| 00:49 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php |
[production] |
| 00:48 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php |
[production] |
| 00:40 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php |
[production] |
| 00:30 |
<Tim> |
restarted socat on fenari so that fatal.log is reopened |
[production] |
| 00:29 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php 'removed logging hack tweaks.' |
[production] |
| 00:29 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php 'logging hack tweaks.' |
[production] |
| 00:27 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php 'removed some temp logging' |
[production] |
| 00:27 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php |
[production] |
| 00:16 |
<aaron> |
synchronized php-1.20wmf2/includes/User.php |
[production] |
| 00:00 |
<binasher> |
pulling cp1044 from lvs for testing |
[production] |