|
2013-03-18
§
|
| 19:07 |
<asher> |
synchronized wmf-config/db-eqiad.php 'returning db1043 db1009' |
[production] |
| 19:04 |
<reedy> |
synchronized docroot |
[production] |
| 19:02 |
<asher> |
synchronized wmf-config/db-eqiad.php 'db1050 for special s1' |
[production] |
| 18:56 |
<asher> |
synchronized wmf-config/db-eqiad.php 'pulling db1043, db1009 for upgrade' |
[production] |
| 18:40 |
<reedy> |
synchronized php-1.21wmf12 'Initial sync of php-1.21wmf12' |
[production] |
| 18:16 |
<awight> |
payments cluster updated from c19cc66 to fe4fb96b |
[production] |
| 18:06 |
<reedy> |
synchronized wmf-config/CommonSettings.php |
[production] |
| 18:03 |
<aaron> |
synchronized wmf-config/PrivateSettings.php 'Removed old testing cruft' |
[production] |
| 16:19 |
<cmjohnson1> |
removing/replacing disk 4 ms-be1004 |
[production] |
| 16:01 |
<hashar> |
Hard restarted Zuul which was dead locked again :-( |
[production] |
| 13:23 |
<paravoid> |
depooled, upgraded, restarted and pooled again all ms-fe[1-4] |
[production] |
| 13:01 |
<paravoid> |
depooling ms-fe1 for new swift version (patched 1.7.4) |
[production] |
| 11:27 |
<apergos> |
restarted both the varnishncsas on niobium, they were giant again |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.21wmf11) at Mon Mar 18 02:30:03 UTC 2013 |
[production] |
|
2013-03-15
§
|
| 23:24 |
<laner> |
synchronized wmf-config/CommonSettings.php 'Disabling aft5' |
[production] |
| 22:45 |
<asher> |
synchronized wmf-config/db-eqiad.php 'returning es1007,10 at lower weights' |
[production] |
| 22:35 |
<binasher> |
rebooted es1007,1010 for kernel upgrades |
[production] |
| 22:33 |
<asher> |
synchronized wmf-config/db-eqiad.php 'temporarily pulling es1007, es1010' |
[production] |
| 22:23 |
<Ryan_Lane> |
hid domain dropdown for account creation and login on wikitech |
[production] |
| 22:23 |
<Ryan_Lane> |
changed wikitech configuration to require email address on account creation |
[production] |
| 22:22 |
<Ryan_Lane> |
deployed change 54093 to wikitech |
[production] |
| 22:03 |
<hashar> |
Zuul : opened a bug about this week slowness, breakage: https://bugzilla.wikimedia.org/show_bug.cgi?id=46176 |
[production] |
| 21:51 |
<aaron> |
synchronized php-1.21wmf11/includes 'deployed 08295adcdc4a4130954f52fb1eb5377638cd97bc' |
[production] |
| 21:36 |
<binasher> |
db59 -> mariadb 5.5.30 |
[production] |
| 21:29 |
<hashar> |
restarted Zuul with the safe version ff79197 |
[production] |
| 21:27 |
<hashar> |
downgrading Zuul to safe version ff79197 (aka stop doing `git remote update` on ref-update events) |
[production] |
| 21:25 |
<binasher> |
added new build of mariadb 5.5.30 to precise-wikimedia on brewster |
[production] |
| 21:20 |
<hashar> |
hard restarted Zuul. It was locked down again :/ |
[production] |
| 21:18 |
<mutante> |
installing package upgrades on gallium |
[production] |
| 21:14 |
<hashar> |
restarting Zuul, it is locked somehow. Definitely need to update it :( |
[production] |
| 21:09 |
<RobH> |
professor services seem back up |
[production] |
| 20:51 |
<RobH> |
professor shutting down for fan swap, short downtime (hopefully) |
[production] |
| 20:47 |
<hashar> |
integration.mediawiki.org is not redirecting to the new integration.wikimedia.org (apache conf is disabled). Patch in {{gerrit|54068}} |
[production] |
| 20:33 |
<mutante> |
puppet runs on neon broken due to mysql puppet issue - mysql-client-5.5 can not create alias mysql-client: object already exists at /var/lib/git/operations/puppet/manifests/mysql.pp:513 |
[production] |
| 20:31 |
<mutante> |
restarting pdns on ns0 |
[production] |
| 20:30 |
<mutante> |
DNS update - removing ipv4/ipv6/ipv6and4/results.labs.wm |
[production] |
| 19:37 |
<mutante> |
maerlant - re-adding 2620:0:862:1::80:2 to eth0, starting nginx |
[production] |
| 19:22 |
<sbernardin> |
taking down raskin for decommissioning |
[production] |
| 19:13 |
<mutante> |
maerlant - dpkg: error processing ganglia-monitor | backing up /var/www/ to tridge /data/other then deleting contents | removing sites-enabled/ipv6and4 | nginx can't bind to [2620:0:862:1::80:2]:443 |
[production] |
| 18:50 |
<mutante> |
installing more package upgrades on maerlant (wikimedia-lvs-realserver, libc6, nginx...) |
[production] |
| 18:46 |
<mutante> |
installing package upgrades on formey |
[production] |
| 18:44 |
<mutante> |
fixing broken dpkg/APT on maerlant - dpkg --configure -a , re-run puppet and let it install package upgrades |
[production] |