151-200 of 2992 results (5ms)
2009-07-16 §
04:24 <Rob> outage for esams servers started at approx 3:20 gmt [production]
04:15 <Rob> still waiting on esams to update us about the rack(s), moving traffic to pmtpa [production]
00:59 <tomaszf> started backup for latest xml snapshots from storage2 to ms4 [production]
2009-07-15 §
22:30 <Rob> updated dns for new snapshot servers becasue tomasz did not want to be in charge of dump servers. [production]
22:10 <brion> brion checking around for 0-byte files (not thumbs) to see if we can recover [production]
21:33 <atglenn> verified that zfs patch is in place on ms4 (it got sucked in during river's update yesterday) [production]
21:26 <brion> synchronized php-1.5/CommonSettings.php 'Restore fancy captcha mode' [production]
21:16 <I_am_not_root> synchronized php-1.5/CommonSettings.php 're-enabling Uploads and removing site notice.' [production]
21:01 <atglenn> rebooting ms1 after applying zfs patch. *cross fingers* [production]
20:51 <brion> synchronized php-1.5/CommonSettings.php [production]
20:51 <brion> synchronized php-1.5/InitialiseSettings.php [production]
20:42 <brion> reenabled captcha in simple mode (no images; math q) [production]
20:37 <brion> captcha system broken while images are offline, need to disable it temporarily [production]
20:18 <brion> updated http://en.wikipedia.org/wiki/MediaWiki:Uploaddisabledtext & http://commons.wikimedia.org/wiki/MediaWiki:Uploaddisabledtext [production]
19:43 <fvassard> synchronized php-1.5/CommonSettings.php 'Disabling Uploads while ms1 gets fixed (again with an s after upload).' [production]
19:40 <fvassard> synchronized php-1.5/CommonSettings.php 'Disabling Uploads while ms1 gets fixed.' [production]
19:40 <atglenn> bringing solaris up to current patch level on ms1 [production]
19:34 <brion> Ok, we're going to temporarily shut off uploading and unmount the uploads dir while we muck about with ms1. [production]
19:14 <brion> dropping export/upload@daily-2009-07-11_03:10:00 [production]
19:08 <brion> restarting web server on ms1, see if that resets some connections to the backend scalers [production]
19:05 <brion> restarting nfsd on ms1 [production]
18:58 <brion> dropping zfs snapshot export/upload@daily-2009-07-09_03:10:00 [production]
18:25 <RobH_A90> drac and physical setup done for dump1,2,3, will install remotely [production]
17:52 <RobH_A90> updated dns for new dump processing servers public and management ips [production]
17:41 <Fred> bounced apache on srv45 [production]
17:37 <Fred> bounced apache on srv47 [production]
17:09 <RobH_A90> pdf1 is not coming back, working on it [production]
16:56 <RobH_A90> shutting down pdf1 and mobile1 to move their power too, weee [production]
16:55 <RobH_A90> shutting down spence to move [production]
16:50 <RobH_A90> shutting down singer to move its power, blogs and other associated services will be offline for approx. 5 minutes [production]
16:47 <Andrew> Restarting apache on prototype [production]
16:46 <RobH_A90> shutting down grosley for power move [production]
16:45 <RobH_A90> all these power moves are to add the new dump processing servers to the rack [production]
16:45 <RobH_A90> shutting down fenari for power move [production]
16:43 <RobH_A90> shut down eiximenis and erzurumi to move their power [production]
16:34 <RobH_A90> shutting down some servers and moving power around in a4-sdtpa [production]
16:19 <Andrew> Test [production]
16:17 <Andrew> Changed morebots to tell you through a channel message instead of a private notice when the logging is successful. [production]
15:54 <Fred> kernel updated on wikitech from 2.6.18.8 to 2.6.29 (latest available on linode) [production]
15:49 <Andrew> Fixed auto-submission of honeypot data, was broken because it needed my perl include path. [production]
15:40 <Fred> rebooting wikitech to install new kernel [production]
2009-07-14 §
23:15 <brion> synchronized php-1.5/InitialiseSettings.php 'fix fix to enwiki confirmed gruop :D' [production]
22:24 <brion> synchronized php-1.5/InitialiseSettings.php 'fix to confirmed group for en' [production]
20:44 <brion> synchronized wmf-deployment/cache/trusted-xff.cdb [production]
20:41 <brion> synchronized wmf-deployment/cache/trusted-xff.cdb [production]
20:40 <brion> synchronized wmf-deployment/AdminSettings.php [production]
20:22 <Fred> restarted a bunch of dead apaches [production]
20:10 <brion> doing a sync-common-all w/ attempt to put test.wikipedia on wmf-deployment branch [production]
19:50 <robh> synchronized php-1.5/InitialiseSettings.php [production]
19:11 <robh> synchronized php-1.5/InitialiseSettings.php 'bug 19611 forgot one thing' [production]