|
2013-05-07
§
|
| 07:59 |
<paravoid> |
partially upgrading ceph to 0.56.6 |
[production] |
| 07:27 |
<Tim> |
on holmium: reverted firewall, too hard |
[production] |
| 07:22 |
<Tim> |
on holmium: added some ad hoc iptables rules intended to keep the rest of the site mostly safe from it overnight |
[production] |
| 06:59 |
<Tim> |
on holmium: chown'd to root all script files writable by www-data and all directories which are configured to allow PHP execution |
[production] |
| 06:27 |
<Tim> |
on marmontel: disabled apache logrotate, stopped apache, disabled apache init script |
[production] |
| 04:19 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue May 7 04:19:39 UTC 2013 |
[production] |
| 02:25 |
<LocalisationUpdate> |
completed (1.22wmf2) at Tue May 7 02:25:11 UTC 2013 |
[production] |
| 02:16 |
<LocalisationUpdate> |
completed (1.22wmf3) at Tue May 7 02:16:06 UTC 2013 |
[production] |
| 01:29 |
<bsitu> |
synchronized php-1.22wmf3/includes/User.php 'Two fixes to User::getNewMessageRevisionId() from kaldari' |
[production] |
| 01:28 |
<reedy> |
synchronized php-1.22wmf3/extensions/WikimediaMaintenance |
[production] |
| 00:38 |
<reedy> |
synchronized php-1.22wmf3/includes/User.php 'Live hack to stop fatals' |
[production] |
| 00:06 |
<bsitu> |
synchronized php-1.22wmf3/extensions/Echo 'Update Echo to master' |
[production] |
| 00:06 |
<bsitu> |
synchronized php-1.22wmf3/docs/hooks.txt 'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' |
[production] |
| 00:06 |
<bsitu> |
synchronized php-1.22wmf3/RELEASE-NOTES-1.22 'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' |
[production] |
| 00:05 |
<bsitu> |
synchronized php-1.22wmf3/includes/WikiPage.php 'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' |
[production] |
| 00:03 |
<bsitu> |
synchronized php-1.22wmf3/includes/User.php 'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' |
[production] |
| 00:02 |
<bsitu> |
synchronized php-1.22wmf3/includes/Skin.php 'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' |
[production] |
| 00:01 |
<bsitu> |
synchronized php-1.22wmf3/includes/OutputPage.php 'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' |
[production] |
|
2013-05-06
§
|
| 23:25 |
<mutante> |
restarting prelabsdbdb on db1054 |
[production] |
| 22:47 |
<mutante> |
apache-graceful-all, add iegcom virtual host |
[production] |
| 21:00 |
<mutante> |
blog is back up |
[production] |
| 20:59 |
<mutante> |
holmium: bring up interface lo, comment "search" line in resolv.conf, restart varnish, ... |
[production] |
| 20:46 |
<paravoid> |
swift replication scripts running on ms-fe1002 under screen -- kill if they have any effect on production |
[production] |
| 20:14 |
<mutante> |
DNS update - sync with ns2 |
[production] |
| 20:04 |
<mutante> |
restarting parsoid on wtp1001 |
[production] |
| 20:02 |
<mutante> |
restarting pdns on ns0 |
[production] |
| 20:00 |
<mutante> |
killing parsoid server.js on wtp1001 |
[production] |
| 19:59 |
<aaron> |
synchronized multiversion/MWMultiVersion.php 'f257ddac39194f8bfcfd145104999fdba83d33f5' |
[production] |
| 19:58 |
<mutante> |
DNS update - switch all blog cnames over to holmium |
[production] |
| 19:29 |
<reedy> |
synchronized php-1.22wmf3/extensions/ZeroRatedMobileAccess/ |
[production] |
| 19:24 |
<LeslieCarr> |
moved gateway from csw1-esams to cr1-esams |
[production] |
| 19:18 |
<Krinkle> |
Reloading Zuul to deploy I259585e78ee |
[production] |
| 19:17 |
<LeslieCarr> |
rebooting ns2 |
[production] |
| 19:15 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 19:13 |
<binasher> |
rebuilt db1016 from hotbackup of db1001; pmtpa side of m1 needs to be rebuilt as well |
[production] |
| 18:41 |
<RobH> |
ns2/nescio appears offline, investigating |
[production] |
| 18:37 |
<RobH> |
ns2 appears offline |
[production] |
| 18:36 |
<RobH> |
authdns-update crashed on ns2, rerunning |
[production] |
| 18:33 |
<RobH> |
holmium |
[production] |
| 18:33 |
<RobH> |
authdns-update for new server homium |
[production] |
| 18:07 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.22wmf3 |
[production] |
| 17:17 |
<LeslieCarr> |
marmontel's interface is now in the sandbox vlan |
[production] |
| 17:15 |
<LeslieCarr> |
turned off marmontel's network port |
[production] |
| 16:11 |
<reedy> |
synchronized docroot |
[production] |
| 13:54 |
<hashar> |
gallium: installed manually libcidr0-dev libanon0-dev . Puppet change is {{gerrit|62434}} |
[production] |
| 09:51 |
<paravoid> |
upgrading Ganglia Web to 3.5.7 |
[production] |
| 08:25 |
<hashar> |
Zuul upgrading to upstream version 7191ee8 + 1 wmf hack. |
[production] |
| 06:45 |
<Nemo_bis> |
Translate search on vanadium keeps being broken https://toolserver.org/~nemobis/tmp/SearchTranslations.log |
[production] |
| 06:14 |
<paravoid> |
pushing new swift rings: ms-be1 weight 50->0, ms-be2 33->75 |
[production] |
| 02:18 |
<paravoid> |
restarting gerrit |
[production] |