|
2013-03-13
§
|
| 21:38 |
<Tim> |
on professor: restarted collector |
[production] |
| 21:21 |
<RobH> |
repooling all ssl servers (except broken ssl3004) in all sites. new unified certificate is now in use. |
[production] |
| 21:15 |
<RobH> |
depooling ssl servers 3 and 4 in each dc site |
[production] |
| 21:14 |
<RobH> |
repooling the first two ssl servers per site |
[production] |
| 21:08 |
<py> |
synchronized wmf-config/lucene-production.php 'temp moving all earch traffic to pmtpa for upgrades in eqiad' |
[production] |
| 21:05 |
<RobH> |
setting first 2 ssl servers per dc site to false in pybal |
[production] |
| 20:21 |
<Ryan_Lane> |
repooling ssl1001 |
[production] |
| 20:20 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Re-enable collection' |
[production] |
| 20:16 |
<Ryan_Lane> |
depooling ssl1001 |
[production] |
| 20:08 |
<LeslieCarr> |
cp1022 unresponsive, power cycling |
[production] |
| 19:44 |
<LeslieCarr> |
switched preferred path between esams and eqiad |
[production] |
| 19:41 |
<LeslieCarr> |
deploying a possibly mobile front end traffic effecting ipv6 apache request firewall filter |
[production] |
| 19:26 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Disable Collection extension' |
[production] |
| 19:08 |
<LeslieCarr> |
site was popedotted ! |
[production] |
| 18:47 |
<py> |
synchronized wmf-config/lucene-production.php 'moving all search traffic back to eqaid' |
[production] |
| 18:31 |
<py> |
synchronized wmf-config/lucene-production.php 'temp moving all earch traffic to pmtpa for upgrades in eqiad' |
[production] |
| 18:15 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable scribunto EVERYWHERE' |
[production] |
| 18:09 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Everything else to 1.21wmf11 |
[production] |
| 17:23 |
<pp-pdf1> |
downgrade to 0.14.3 |
[production] |
| 16:39 |
<pp-pdf1> |
upgraded mwlib to 0.15.1. |
[production] |
| 16:32 |
<apergos> |
cleaned up borked puppet agent on ms-be12 and re-enabled/restarted it |
[production] |
| 12:30 |
<anomie> |
synchronized php-1.21wmf11/extensions/Scribunto/common/Hooks.php 'Fix for bug 46031' |
[production] |
| 12:30 |
<anomie> |
synchronized php-1.21wmf11/extensions/Scribunto/common/Common.php 'Fix for bug 46031' |
[production] |
| 12:07 |
<Danny_B> |
reenabled mail delivery to wikibugs 30 mins ago (disabled automatically for excessive bounces) |
[production] |
| 11:42 |
<apergos> |
restarted morebots |
[production] |
| 01:10 |
<Tim> |
running updateCollation.php on all uca-* wikis in 6 screens on hume |
[production] |
| 00:44 |
<dzahn> |
gracefulled all apaches |
[production] |
| 00:39 |
<Ryan_Lane> |
depooling ssl1 ssl2 ssl1001 ssl1002 ssl3001 ssl3002 |
[production] |
| 00:35 |
<Ryan_Lane> |
pooling ssl3 ssl4 ssl1003 ssl1004 ssl3003 |
[production] |
| 00:31 |
<Ryan_Lane> |
depooling ssl3 ssl4 ssl1003 ssl1004 ssl3003 |
[production] |
| 00:30 |
<Ryan_Lane> |
pooling ssl1 ssl2 ssl1001 ssl1002 ssl3001 ssl3002 |
[production] |
| 00:16 |
<Ryan_Lane> |
depooling ssl1 ssl2 ssl1001 ssl1002 ssl3001 ssl3002 |
[production] |
| 00:12 |
<Tim> |
manually creating missing /etc/apache2/wmf symlinks on servers that lack them |
[production] |
| 00:07 |
<dzahn> |
gracefulled all apaches |
[production] |
|
2013-03-12
§
|
| 23:39 |
<Tim> |
updating PHP to php5-3.5+wmf1 for new ICU |
[production] |
| 23:39 |
<notpeter> |
upgrading pmtpa search nodes to lucene-search-2 2.1.7wm1 |
[production] |
| 23:34 |
<tstarling> |
synchronized php-1.21wmf11/maintenance/updateCollation.php |
[production] |
| 23:33 |
<tstarling> |
synchronized php-1.21wmf10/maintenance/updateCollation.php |
[production] |
| 22:33 |
<Ryan_Lane> |
pooling ssl3003 ssl3 ssl4 ssl1003 ssl1004 |
[production] |
| 22:28 |
<Ryan_Lane> |
depooling ssl3003 ssl3 ssl4 ssl1003 ssl1004 |
[production] |
| 22:28 |
<Ryan_Lane> |
pooling ssl3001 ssl3002 ssl1 ssl2 ssl1001 and ssl1002 |
[production] |
| 22:23 |
<mutante> |
pushing new version 2.7 of wikimedia-task-appserver to wikipedia-precise repo (to fix apache-sanity-check) |
[production] |
| 22:21 |
<Ryan_Lane> |
depooling ssl3001 ssl3002 ssl1 ssl2 ssl1001 and ssl1002 |
[production] |
| 21:23 |
<Ryan_Lane> |
seems that the unified certificate is missing mediawiki.org, depooling the ssl servers that are serving it |
[production] |
| 21:22 |
<Ryan_Lane> |
installing the new unified ssl certificate on all ssl servers |
[production] |
| 19:20 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 18:54 |
<RobH> |
authdns-update for europium to gadolinium ip |
[production] |
| 18:47 |
<RobH> |
europium shuttin down |
[production] |
| 18:43 |
<mark> |
Imported ganglia 3.5.0 packages into the precise-wikimedia APT repository |
[production] |
| 18:19 |
<RobH> |
europium reclaimed per rt4689 |
[production] |