|
2009-06-26
§
|
| 20:33 |
<Rob> |
blog.wikimedia.org back online, very restrictive settings and improved security (i hope) |
[production] |
| 19:09 |
<Rob> |
singer https services back online for survey. ocs, wm09schols |
[production] |
| 18:20 |
<Rob> |
Singer Restore: survey.wikimedia.org - UP, ocs.wikimania2009.wikimedia.org - UP |
[production] |
| 17:11 |
<Rob> |
pulled blog.wikimedia.org out of squid via changing its dns to point directly at singer. just to make it easier to secure and fix on the fly later today |
[production] |
| 15:03 |
<Rob> |
firing singer back up for reinstall and restoration from the haxor |
[production] |
| 14:42 |
<midom> |
synchronized php-1.5/includes/parser/ParserCache.php 'removing the pcache hack for now' |
[production] |
| 11:44 |
<domas> |
halted singer, don't start it without contacting me. |
[production] |
| 07:27 |
<brion> |
synchronized php-1.5/includes/parser/ParserCache.php 'Putting live hack for MJ article back. Total traffic spike down but article traffic still spiking.' |
[production] |
| 05:53 |
<Tim> |
rebooting srv182, swap death since ~00:10 UTC |
[production] |
| 05:50 |
<Tim> |
killed broken mysqld_safe instances on srv157, srv161, srv167, srv173, srv174, srv175, srv181, srv185, srv186 |
[production] |
| 05:17 |
<jeluf> |
synchronized php-1.5/mc-pmtpa.php 'replace srv113 by srv142' |
[production] |
| 05:14 |
<jeluf> |
synchronized php-1.5/mc-pmtpa.php 'replace srv182 by srv92' |
[production] |
|
2009-06-25
§
|
| 23:36 |
<brion> |
synchronized php-1.5/includes/parser/ParserCache.php 'live hack to extend caching of Michael Jackson' |
[production] |
| 23:17 |
<brion> |
synchronized php-1.5/includes/db/Database.php 'tweak the db err msg' |
[production] |
| 22:46 |
<brion> |
synchronized php-1.5/mc-pmtpa.php 'putting 156 mc back' |
[production] |
| 22:44 |
<brion> |
synchronized php-1.5/mc-pmtpa.php 'swapping in 145 in place of temp down 156' |
[production] |
| 22:43 |
<tomaszf> |
rebooting srv156 due to hard down |
[production] |
| 22:42 |
<midom> |
synchronized php-1.5/db.php '156 down' |
[production] |
| 18:26 |
<RobH_A90> |
srv92 back up with replaced fans |
[production] |
| 18:16 |
<RobH_A90> |
srv92 down due to bad fan, going to replace it with another dead server fan, yay for frankenstien servers |
[production] |
| 18:09 |
<RobH_A90> |
db17 has had a cold reset, service processors are now responsive, as well as system |
[production] |
| 09:21 |
<tstarling> |
synchronized php-1.5/languages/Language.php |
[production] |
|
2009-06-24
§
|
| 16:27 |
<robh> |
synchronized php-1.5/InitialiseSettings.php 'bug 19364' |
[production] |
| 15:56 |
<Fred> |
modified /home/wikipedia/conf/nagios/sync to include new nagios server in the sync/restart process. |
[production] |
| 15:55 |
<Fred> |
cleaned up /home/wikipedia/conf/nagios/conf.php to remove unused server / conf parameters. |
[production] |
| 15:54 |
<Fred> |
Cleaned up dsh node_group and resynched Nagios. |
[production] |
| 10:26 |
<midom> |
synchronized php-1.5/includes/specials/SpecialRecentchanges.php 'livehack upper limit, as mediawiki devs are lazy.. oh wait.' |
[production] |
| 08:29 |
<domas> |
compressed databases on db3-5 using lzip, freed up the space. :) |
[production] |
| 08:16 |
<midom> |
synchronized php-1.5/db.php 'db24 back to life' |
[production] |
| 06:28 |
<Tim> |
updated GeSHi to 1.0.8.4 |
[production] |
| 05:54 |
<Tim> |
removed all SVN externals from the MW working copy. Updated extensions/SyntaxHighlight_GeSHi to r52346. Scapping. |
[production] |
|
2009-06-23
§
|
| 22:00 |
<Fred> |
make && deploy of new squid configuration with added acl for spence.w.o |
[production] |
| 21:07 |
<robh> |
synchronized php-1.5/InitialiseSettings.php '19364 Set project namespace of Portuguese Wikibooks' |
[production] |
| 20:49 |
<robh> |
synchronized php-1.5/InitialiseSettings.php 'Fixing issues with arwiki namespacing and formatting' |
[production] |
| 19:33 |
<Rob> |
checked up on the updateTitles running against enwiki on hume. On 11864380 / 23266362 |
[production] |
| 19:31 |
<robh> |
synchronized php-1.5/InitialiseSettings.php '19357 Create new namespace on arwiki' |
[production] |
| 19:22 |
<robh> |
synchronized php-1.5/InitialiseSettings.php |
[production] |
| 19:14 |
<robh> |
synchronized php-1.5/InitialiseSettings.php 'Bug 19116 adding namespace aliases for itwikibooks' |
[production] |
| 15:39 |
<robh> |
synchronized php-1.5/InitialiseSettings.php '19328 set for fywikibooks - typosssss' |
[production] |
| 15:36 |
<robh> |
synchronized php-1.5/InitialiseSettings.php '19328 set for fywikibooks' |
[production] |
| 15:24 |
<robh> |
synchronized php-1.5/abusefilter.php 'Updated with Andrew' |
[production] |
| 15:15 |
<Rob> |
wrong reason listed in sync, opps, was for bug 19272 |
[production] |