|
2010-05-26
§
|
| 23:59 |
<Rob> |
updated blog to vector earlier today, didnt log... just pushed another update to the theme, blog is still happah |
[production] |
| 21:43 |
<robh> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'updated foundationwiki default skin from monobook to vector' |
[production] |
| 17:02 |
<Tim> |
on mobile1 nginx: set proxy_next_upstream=off to avoid having the site go down regularly for 10s at a time |
[production] |
| 16:30 |
<Tim> |
changing mobile1 nginx log format |
[production] |
| 06:39 |
<jeluf> |
synchronized php-1.5/wmf-config/InitialiseSettings.php '23667 - Local time for Macedonian Wikipedia' |
[production] |
| 03:39 |
<Tim> |
set up SSLCertificateChainFile on williams (otrs) per bug 23631, used the cut down version of gd_bundle.crt posted on the bug report |
[production] |
| 02:08 |
<andrew> |
synchronized php-1.5/includes/LogEventsList.php |
[production] |
| 02:07 |
<andrew> |
synchronized php-1.5/includes/specials/SpecialRevisiondelete.php |
[production] |
| 02:07 |
<andrew> |
synchronized php-1.5/languages/messages/MessagesEn.php |
[production] |
| 02:05 |
<Andrew> |
Deploying r66793 and r66823, first to test.wikipedia.org for final testing, then to all sites. |
[production] |
|
2010-05-25
§
|
| 17:02 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Re-add style version appendix now that borked usability JS was purged from Varnish' |
[production] |
| 12:51 |
<domas> |
what happened - server goes down, configuration gets updated, server comes up, builds new cache out of old conf, sync has timestamp older than cache, mw has stale config |
[production] |
| 12:47 |
<domas> |
cleaned /tmp/mw-cache for some servers - apparently sync-common is not enough to get stuff in sync anymore :) we have now monobook pages in caches, I guess. |
[production] |
| 12:26 |
<rainman-sr> |
freed up some space and restarted indexing on searchidx1, indexing was stuck because the disk was full |
[production] |
| 11:16 |
<mark> |
Downgraded puppet on db36 |
[production] |
| 06:56 |
<domas> |
resynced srv206/srv193 - seems to have lost critical configuration updates while they were down. Again, why servers are not resynced after downtime? |
[production] |
| 01:24 |
<Tim> |
fixed redirect from http://racktables to https://racktables, was broken due to IP address change |
[production] |
| 01:15 |
<tstarling> |
synchronized php-1.5/wmf-config/CommonSettings.php 'removed cruft from wgStyleVersion' |
[production] |
| 01:12 |
<tstarling> |
synchronized php-1.5/skins/common/wikibits.js |
[production] |
| 00:19 |
<aaron> |
synchronized php-1.5/wmf-config/flaggedrevs.php 'removed Editor group from flaggedrevs_labswikimedia' |
[production] |
| 00:02 |
<AaronSchulz> |
unhide the username of edits by "The Thing That Should Not Be" after accidental account locking (and unlock didn't work) |
[production] |
|
2010-05-24
§
|
| 15:01 |
<Rob> |
sq39 will stay offline for a bit, have a configuration concern with replacement disk performance. |
[production] |
| 15:01 |
<Rob> |
pushed updates to srv206 and srv193, they should be back online and ok now. |
[production] |
| 14:54 |
<Rob> |
sq39 has disk replaced and reinstalled, but its failing squid backend due to configuration. pulling out of rotation to look at later today. |
[production] |
| 14:51 |
<Rob> |
working on srv206 and srv193 |
[production] |
| 14:21 |
<Rob> |
replaced bad disk in sq39, reinstalling. |
[production] |
| 12:16 |
<domas> |
db32, db36, db38 and db40 are now serving enwiki (+ db12 and db26 ) |
[production] |
| 12:15 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 09:25 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'db22 being removed too' |
[production] |
| 09:23 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'removing db14, db16 from enwiki duty, will host different clusters, db26 upgrade' |
[production] |
| 09:17 |
<domas> |
master switch db16->db36, new master position - db36-bin.000001:106 |
[production] |
| 09:16 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 09:16 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 09:09 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 09:06 |
<domas> |
srv141 was in prod, but not in mw_inst |
[production] |
| 07:24 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 07:09 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 07:06 |
<domas> |
srv134 was having i/o issues, probably bad disk, rebooted for it to stop flapping, sorry Jens :) |
[production] |