|
2009-07-15
§
|
| 22:30 |
<Rob> |
updated dns for new snapshot servers becasue tomasz did not want to be in charge of dump servers. |
[production] |
| 22:10 |
<brion> |
brion checking around for 0-byte files (not thumbs) to see if we can recover |
[production] |
| 21:33 |
<atglenn> |
verified that zfs patch is in place on ms4 (it got sucked in during river's update yesterday) |
[production] |
| 21:26 |
<brion> |
synchronized php-1.5/CommonSettings.php 'Restore fancy captcha mode' |
[production] |
| 21:16 |
<I_am_not_root> |
synchronized php-1.5/CommonSettings.php 're-enabling Uploads and removing site notice.' |
[production] |
| 21:01 |
<atglenn> |
rebooting ms1 after applying zfs patch. *cross fingers* |
[production] |
| 20:51 |
<brion> |
synchronized php-1.5/CommonSettings.php |
[production] |
| 20:51 |
<brion> |
synchronized php-1.5/InitialiseSettings.php |
[production] |
| 20:42 |
<brion> |
reenabled captcha in simple mode (no images; math q) |
[production] |
| 20:37 |
<brion> |
captcha system broken while images are offline, need to disable it temporarily |
[production] |
| 20:18 |
<brion> |
updated http://en.wikipedia.org/wiki/MediaWiki:Uploaddisabledtext & http://commons.wikimedia.org/wiki/MediaWiki:Uploaddisabledtext |
[production] |
| 19:43 |
<fvassard> |
synchronized php-1.5/CommonSettings.php 'Disabling Uploads while ms1 gets fixed (again with an s after upload).' |
[production] |
| 19:40 |
<fvassard> |
synchronized php-1.5/CommonSettings.php 'Disabling Uploads while ms1 gets fixed.' |
[production] |
| 19:40 |
<atglenn> |
bringing solaris up to current patch level on ms1 |
[production] |
| 19:34 |
<brion> |
Ok, we're going to temporarily shut off uploading and unmount the uploads dir while we muck about with ms1. |
[production] |
| 19:14 |
<brion> |
dropping export/upload@daily-2009-07-11_03:10:00 |
[production] |
| 19:08 |
<brion> |
restarting web server on ms1, see if that resets some connections to the backend scalers |
[production] |
| 19:05 |
<brion> |
restarting nfsd on ms1 |
[production] |
| 18:58 |
<brion> |
dropping zfs snapshot export/upload@daily-2009-07-09_03:10:00 |
[production] |
| 18:25 |
<RobH_A90> |
drac and physical setup done for dump1,2,3, will install remotely |
[production] |
| 17:52 |
<RobH_A90> |
updated dns for new dump processing servers public and management ips |
[production] |
| 17:41 |
<Fred> |
bounced apache on srv45 |
[production] |
| 17:37 |
<Fred> |
bounced apache on srv47 |
[production] |
| 17:09 |
<RobH_A90> |
pdf1 is not coming back, working on it |
[production] |
| 16:56 |
<RobH_A90> |
shutting down pdf1 and mobile1 to move their power too, weee |
[production] |
| 16:55 |
<RobH_A90> |
shutting down spence to move |
[production] |
| 16:50 |
<RobH_A90> |
shutting down singer to move its power, blogs and other associated services will be offline for approx. 5 minutes |
[production] |
| 16:47 |
<Andrew> |
Restarting apache on prototype |
[production] |
| 16:46 |
<RobH_A90> |
shutting down grosley for power move |
[production] |
| 16:45 |
<RobH_A90> |
all these power moves are to add the new dump processing servers to the rack |
[production] |
| 16:45 |
<RobH_A90> |
shutting down fenari for power move |
[production] |
| 16:43 |
<RobH_A90> |
shut down eiximenis and erzurumi to move their power |
[production] |
| 16:34 |
<RobH_A90> |
shutting down some servers and moving power around in a4-sdtpa |
[production] |
| 16:19 |
<Andrew> |
Test |
[production] |
| 16:17 |
<Andrew> |
Changed morebots to tell you through a channel message instead of a private notice when the logging is successful. |
[production] |
| 15:54 |
<Fred> |
kernel updated on wikitech from 2.6.18.8 to 2.6.29 (latest available on linode) |
[production] |
| 15:49 |
<Andrew> |
Fixed auto-submission of honeypot data, was broken because it needed my perl include path. |
[production] |
| 15:40 |
<Fred> |
rebooting wikitech to install new kernel |
[production] |
|
2009-07-14
§
|
| 23:15 |
<brion> |
synchronized php-1.5/InitialiseSettings.php 'fix fix to enwiki confirmed gruop :D' |
[production] |
| 22:24 |
<brion> |
synchronized php-1.5/InitialiseSettings.php 'fix to confirmed group for en' |
[production] |
| 20:44 |
<brion> |
synchronized wmf-deployment/cache/trusted-xff.cdb |
[production] |
| 20:41 |
<brion> |
synchronized wmf-deployment/cache/trusted-xff.cdb |
[production] |
| 20:40 |
<brion> |
synchronized wmf-deployment/AdminSettings.php |
[production] |
| 20:22 |
<Fred> |
restarted a bunch of dead apaches |
[production] |
| 20:10 |
<brion> |
doing a sync-common-all w/ attempt to put test.wikipedia on wmf-deployment branch |
[production] |
| 19:50 |
<robh> |
synchronized php-1.5/InitialiseSettings.php |
[production] |
| 19:11 |
<robh> |
synchronized php-1.5/InitialiseSettings.php 'bug 19611 forgot one thing' |
[production] |
| 19:09 |
<robh> |
synchronized php-1.5/InitialiseSettings.php 'bug 19611' |
[production] |
| 19:08 |
<robh> |
synchronized php-1.5/InitialiseSettings.php 'bug 19611' |
[production] |
| 14:16 |
<domas> |
dropped all june snapshots on ms1, thus providing some relief |
[production] |