|
2012-01-31
§
|
| 19:57 |
<notpeter> |
restarting puppetmaster proc as it's serving up 500s to all clients (well, 3 randomly selected ones...) |
[production] |
| 18:55 |
<Ryan_Lane> |
restarted squid and lighttpd on brewster |
[production] |
| 18:36 |
<RoanKattouw> |
IRC breakage postmortem: MediaWiki was configured to send UDP packets to .179 (ekrem-old) instead of .178 (ekrem) |
[production] |
| 18:32 |
<catrope> |
synchronized wmf-config/InitialiseSettings.php 'Change wgRC2UDPAddress to the new ekrem IP' |
[production] |
| 16:56 |
<mutante> |
restarted ircd on ekrem once again because we still cant join channels .. problem remains |
[production] |
| 16:52 |
<mutante> |
17:52 [Users #en.wikipedia] |
[production] |
| 16:35 |
<mutante> |
restarted IRC bot on ekrem (needs dependency to start after ircd) |
[production] |
| 16:30 |
<mutante> |
ekrem - gets Error 500 on SERVER when running puppet |
[production] |
| 16:30 |
<reedy> |
synchronized php-1.18/extensions/SpamBlacklist/SpamBlacklist_body.php '[[rev:110401|r110401]]' |
[production] |
| 16:28 |
<mutante> |
ekrem - su -c /usr/local/ircd-ratbox/bin/ircd irc |
[production] |
| 16:21 |
<mutante> |
powercycling ekrem - mgmt just showed "Stopping web" and was frozen completely |
[production] |
| 16:17 |
<RoanKattouw> |
ekrem suddenly died around 16:03 UTC, breaking the RC IRC feed |
[production] |
| 15:06 |
<mutante> |
changed nameservers for wikimedia.pl per [[RT:2277]]/[[bugzilla:33509]] |
[production] |
| 09:28 |
<catrope> |
synchronized php-1.18/includes/Wiki.php '[[rev:110368|r110368]]' |
[production] |
| 09:27 |
<catrope> |
synchronized php-1.18/includes/Exception.php '[[rev:110368|r110368]]' |
[production] |
| 06:34 |
<Tim> |
added myself to the gerrit "administrators" group |
[production] |
| 05:23 |
<Tim> |
the segfaults didn't stop, so I'm disabling wmerrors entirely for now |
[production] |
| 05:13 |
<Tim> |
since puppet is broken, disabled wmerrors backtrace logging by adding a separate configuration file in /etc/php5/conf.d and reloading apache |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.18) at Tue Jan 31 02:06:09 UTC 2012 |
[production] |
|
2012-01-30
§
|
| 23:56 |
<awjr> |
synchronized i18n files for DonationInterface on payments cluster to [[rev:110342|r110342]] |
[production] |
| 23:46 |
<Ryan_Lane> |
moving instances from virt2 to virt1 to rebalance compute cluster |
[production] |
| 19:48 |
<awjrichards> |
synchronizing Wikimedia installation... : Syncing CentralNotice to [[rev:110026|r110026]] of trunk, includes important fix for 1.19 compatibility |
[production] |
| 19:28 |
<asher> |
synchronized wmf-config/db.php 'raising db54 weight' |
[production] |
| 18:57 |
<asher> |
synchronized wmf-config/db.php 'adding db54 to s2' |
[production] |
| 18:39 |
<mutante> |
running authdns-update to activate be.wikimedia.org |
[production] |
| 18:19 |
<nikerabbit> |
synchronized php-1.18/extensions/Narayam/resources/ext.narayam.rules.as.js 'I18ndeploy [[rev:110311|r110311]] - [[bugzilla:33924|bug 33924]]' |
[production] |
| 18:17 |
<nikerabbit> |
synchronized php-1.18/extensions/WebFonts/resources/ext.webfonts.fontlist.js 'I18ndeploy [[rev:110311|r110311]] - [[bugzilla:33599|bug 33599]]' |
[production] |
| 18:16 |
<nikerabbit> |
synchronized php-1.18/extensions/Translate/ 'I18ndeploy [[rev:110310|r110310]] - Translate help links' |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.18) at Mon Jan 30 02:06:42 UTC 2012 |
[production] |
|
2012-01-27
§
|
| 21:57 |
<pgehres> |
updates complete, re-enabling queue consumption on jenkins on aluminium |
[production] |
| 21:42 |
<pgehres> |
pausing payments queue consumption in jenkins to backup and then run some db updates |
[production] |
| 21:34 |
<LeslieCarr> |
applying loopback filter on cr1-eqiad |
[production] |
| 21:28 |
<RobH> |
dns update |
[production] |
| 20:53 |
<^demon> |
gallium: clearing /tmp yet again. Aaron claims he's fixing it now |
[production] |
| 19:37 |
<RobH> |
reinstalling sq31 |
[production] |
| 18:32 |
<RobH> |
dns update for fluorine host |
[production] |
| 17:15 |
<binasher> |
s2 dbs are a sad lot. streaming hotback of db1034 to db54 to build a new slave |
[production] |
| 14:37 |
<Jeff_Green> |
dist-upgrading storage3 |
[production] |
| 13:36 |
<^demon> |
gallium: cleaning up /tmp again, tests really need to clean up after themselves. |
[production] |
| 11:34 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'th wikilogos' |
[production] |
| 11:29 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'th wikilogos' |
[production] |
| 11:18 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 33862 - Request for logo change in Tamil Wikiquote' |
[production] |
| 11:11 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 33960 - Import sources for etwiki, etwikisource and etwiktionary' |
[production] |
| 11:07 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 33864 - Flood flag on sr.wiki' |
[production] |
| 02:05 |
<LocalisationUpdate> |
completed (1.18) at Fri Jan 27 02:05:24 UTC 2012 |
[production] |