251-300 of 3070 results (5ms)
2009-07-15 §
18:25 <RobH_A90> drac and physical setup done for dump1,2,3, will install remotely [production]
17:52 <RobH_A90> updated dns for new dump processing servers public and management ips [production]
17:41 <Fred> bounced apache on srv45 [production]
17:37 <Fred> bounced apache on srv47 [production]
17:09 <RobH_A90> pdf1 is not coming back, working on it [production]
16:56 <RobH_A90> shutting down pdf1 and mobile1 to move their power too, weee [production]
16:55 <RobH_A90> shutting down spence to move [production]
16:50 <RobH_A90> shutting down singer to move its power, blogs and other associated services will be offline for approx. 5 minutes [production]
16:47 <Andrew> Restarting apache on prototype [production]
16:46 <RobH_A90> shutting down grosley for power move [production]
16:45 <RobH_A90> all these power moves are to add the new dump processing servers to the rack [production]
16:45 <RobH_A90> shutting down fenari for power move [production]
16:43 <RobH_A90> shut down eiximenis and erzurumi to move their power [production]
16:34 <RobH_A90> shutting down some servers and moving power around in a4-sdtpa [production]
16:19 <Andrew> Test [production]
16:17 <Andrew> Changed morebots to tell you through a channel message instead of a private notice when the logging is successful. [production]
15:54 <Fred> kernel updated on wikitech from 2.6.18.8 to 2.6.29 (latest available on linode) [production]
15:49 <Andrew> Fixed auto-submission of honeypot data, was broken because it needed my perl include path. [production]
15:40 <Fred> rebooting wikitech to install new kernel [production]
2009-07-14 §
23:15 <brion> synchronized php-1.5/InitialiseSettings.php 'fix fix to enwiki confirmed gruop :D' [production]
22:24 <brion> synchronized php-1.5/InitialiseSettings.php 'fix to confirmed group for en' [production]
20:44 <brion> synchronized wmf-deployment/cache/trusted-xff.cdb [production]
20:41 <brion> synchronized wmf-deployment/cache/trusted-xff.cdb [production]
20:40 <brion> synchronized wmf-deployment/AdminSettings.php [production]
20:22 <Fred> restarted a bunch of dead apaches [production]
20:10 <brion> doing a sync-common-all w/ attempt to put test.wikipedia on wmf-deployment branch [production]
19:50 <robh> synchronized php-1.5/InitialiseSettings.php [production]
19:11 <robh> synchronized php-1.5/InitialiseSettings.php 'bug 19611 forgot one thing' [production]
19:09 <robh> synchronized php-1.5/InitialiseSettings.php 'bug 19611' [production]
19:08 <robh> synchronized php-1.5/InitialiseSettings.php 'bug 19611' [production]
14:16 <domas> dropped all june snapshots on ms1, thus providing some relief [production]
01:52 <river> patched ms4 in preperation for upload copy [production]
2009-07-13 §
21:31 <Rob> pushing dns update to fix management ips for new apaches [production]
19:05 <Fred> added storage3 to ganglia monitor. [production]
18:50 <brion> synchronized php-1.5/abusefilter.php 'Disable dewiki missingsummary, mysteriously in abusefilter section. Per bug 19208' [production]
16:30 <Fred> install wikimedia-nis-client to srv66 and mounted /home. [production]
16:28 <brion> synchronized php-1.5/InitialiseSettings.php 'fixing wikispecies RC-IRC prefix to species.wikimedia' [production]
16:27 <brion> test wiki was apparently moved from dead srv35 to srv66, which has new NFS-less config. thus fail since test runs from nfs [production]
16:24 <brion> test wiki borked; reported down for several days now :) investigating [production]
15:12 <midom> synchronized php-1.5/db.php 'db26 raid issues' [production]
14:55 <midom> synchronized php-1.5/db.php 'db3 and db5 coming live as commons servers' [production]
14:13 <domas> dropped few more snapshots, as %sys was increasing on ms1... [production]
11:16 <domas> manually restarted plethora of failing apaches (direct segfaults and other possible APC corruptions, leading to php OOM errors) [production]
09:50 <tstarling> synchronized php-1.5/includes/specials/SpecialBlockip.php [production]
09:00 <Tim> restarted apache2 on image scalers [production]
08:39 <tstarling> synchronized php-1.5/includes/Math.php 'statless render hack' [production]
08:05 <Tim> killed all image scalers to see if that helps with ms1 load [production]
08:00 <Tim> killed waiting apache processes [production]
07:35 <midom> synchronized php-1.5/mc-pmtpa.php [production]
07:24 <midom> synchronized php-1.5/mc-pmtpa.php 'swapping out srv81' [production]