|
2010-05-28
§
|
| 09:00 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'moving over db16 to commons duty, db14 to dewiki, reenabling db4 and db5' |
[production] |
| 07:53 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'removing db5 and db4 for copies' |
[production] |
| 07:11 |
<tstarling> |
synchronized php-1.5/includes/Sanitizer.php 'r66994' |
[production] |
| 06:31 |
<domas> |
upgrading LOM/BIOS/OS on db14,db16,db22 to 3.0/10.4 |
[production] |
| 02:41 |
<Tim> |
srv92 locked up for a few minutes with syslog showing SATA errors. May be near death. |
[production] |
| 01:17 |
<tstarling> |
synchronized php-1.5/wmf-config/CommonSettings.php 're-enabled CategoryTree, unless there is some emergency, you can't just disable extensions like this' |
[production] |
|
2010-05-27
§
|
| 21:35 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Disable CollapsibleNav on outreachwiki' |
[production] |
| 21:22 |
<aaron> |
synchronized php-1.5/wmf-config/flaggedrevs.php |
[production] |
| 21:09 |
<AaronSchulz> |
deployed r66980 (Flaggedrevs_alpha) |
[production] |
| 21:07 |
<aaron> |
synchronizing Wikimedia installation... Revision: 66620 |
[production] |
| 20:24 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Enable Vector and usability features on outreachwiki and foundationwiki' |
[production] |
| 20:17 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Enable CollapsibleNav for beta users on all wikis' |
[production] |
| 20:17 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Enable SimpleSearch for beta users on all wikis' |
[production] |
| 11:59 |
<domas> |
opened a new sub-categories related bug at https://bugzilla.wikimedia.org/show_bug.cgi?id=23682 ;-) |
[production] |
| 11:30 |
<midom> |
synchronized php-1.5/wmf-config/CommonSettings.php 'disable category tree' |
[production] |
| 04:13 |
<aaron> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'vector as default skin for flaggedrevs_labswikimeida' |
[production] |
| 02:40 |
<Tim> |
ran namespaceDupes.php with suffix on jawiki to fix talk namespace duplicates |
[production] |
|
2010-05-26
§
|
| 23:59 |
<Rob> |
updated blog to vector earlier today, didnt log... just pushed another update to the theme, blog is still happah |
[production] |
| 21:43 |
<robh> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'updated foundationwiki default skin from monobook to vector' |
[production] |
| 17:02 |
<Tim> |
on mobile1 nginx: set proxy_next_upstream=off to avoid having the site go down regularly for 10s at a time |
[production] |
| 16:30 |
<Tim> |
changing mobile1 nginx log format |
[production] |
| 06:39 |
<jeluf> |
synchronized php-1.5/wmf-config/InitialiseSettings.php '23667 - Local time for Macedonian Wikipedia' |
[production] |
| 03:39 |
<Tim> |
set up SSLCertificateChainFile on williams (otrs) per bug 23631, used the cut down version of gd_bundle.crt posted on the bug report |
[production] |
| 02:08 |
<andrew> |
synchronized php-1.5/includes/LogEventsList.php |
[production] |
| 02:07 |
<andrew> |
synchronized php-1.5/includes/specials/SpecialRevisiondelete.php |
[production] |
| 02:07 |
<andrew> |
synchronized php-1.5/languages/messages/MessagesEn.php |
[production] |
| 02:05 |
<Andrew> |
Deploying r66793 and r66823, first to test.wikipedia.org for final testing, then to all sites. |
[production] |
|
2010-05-25
§
|
| 17:02 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Re-add style version appendix now that borked usability JS was purged from Varnish' |
[production] |
| 12:51 |
<domas> |
what happened - server goes down, configuration gets updated, server comes up, builds new cache out of old conf, sync has timestamp older than cache, mw has stale config |
[production] |
| 12:47 |
<domas> |
cleaned /tmp/mw-cache for some servers - apparently sync-common is not enough to get stuff in sync anymore :) we have now monobook pages in caches, I guess. |
[production] |
| 12:26 |
<rainman-sr> |
freed up some space and restarted indexing on searchidx1, indexing was stuck because the disk was full |
[production] |
| 11:16 |
<mark> |
Downgraded puppet on db36 |
[production] |
| 06:56 |
<domas> |
resynced srv206/srv193 - seems to have lost critical configuration updates while they were down. Again, why servers are not resynced after downtime? |
[production] |
| 01:24 |
<Tim> |
fixed redirect from http://racktables to https://racktables, was broken due to IP address change |
[production] |
| 01:15 |
<tstarling> |
synchronized php-1.5/wmf-config/CommonSettings.php 'removed cruft from wgStyleVersion' |
[production] |
| 01:12 |
<tstarling> |
synchronized php-1.5/skins/common/wikibits.js |
[production] |
| 00:19 |
<aaron> |
synchronized php-1.5/wmf-config/flaggedrevs.php 'removed Editor group from flaggedrevs_labswikimedia' |
[production] |
| 00:02 |
<AaronSchulz> |
unhide the username of edits by "The Thing That Should Not Be" after accidental account locking (and unlock didn't work) |
[production] |
|
2010-05-24
§
|
| 15:01 |
<Rob> |
sq39 will stay offline for a bit, have a configuration concern with replacement disk performance. |
[production] |
| 15:01 |
<Rob> |
pushed updates to srv206 and srv193, they should be back online and ok now. |
[production] |
| 14:54 |
<Rob> |
sq39 has disk replaced and reinstalled, but its failing squid backend due to configuration. pulling out of rotation to look at later today. |
[production] |
| 14:51 |
<Rob> |
working on srv206 and srv193 |
[production] |
| 14:21 |
<Rob> |
replaced bad disk in sq39, reinstalling. |
[production] |
| 12:16 |
<domas> |
db32, db36, db38 and db40 are now serving enwiki (+ db12 and db26 ) |
[production] |
| 12:15 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 09:25 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'db22 being removed too' |
[production] |
| 09:23 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'removing db14, db16 from enwiki duty, will host different clusters, db26 upgrade' |
[production] |
| 09:17 |
<domas> |
master switch db16->db36, new master position - db36-bin.000001:106 |
[production] |
| 09:16 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |
| 09:16 |
<midom> |
synchronized php-1.5/wmf-config/db.php |
[production] |