751-800 of 10000 results (19ms)
2012-08-16 §
18:14 <aaron> synchronized wmf-config/swift.php 'Disabled redundant thumb copy/purge hooks.' [production]
18:01 <RobH> authdns-update for new es1005-1010 [production]
17:59 <aaron> synchronized wmf-config/filebackend.php 'Make internal reads come from swift for all wikis' [production]
17:50 <apergos> powercycled ms-be3 ... in theory [production]
16:52 <cmjohnson1> shutting down mw18 and mw19 to replace DIMM [production]
16:29 <ottomata> Stopping puppet on brewster again. Sigh. PARTMAAAAN! [production]
16:22 <reedy> synchronized php-1.20wmf9/includes/QueryPage.php [production]
16:07 <cmjohnson1> shutting down mw8 to replace bad DIMM [production]
15:59 <ottomata> starting puppet on brewster. (Woo, partman looking better!) [production]
14:15 <ottomata> stopping puppet on brewster to continue partman troubleshooting for analytics dells [production]
13:26 <mark> Enabled and started SIS deduplication on home_pmtpa on nas1-a [production]
12:05 <mark> Removed OSPF metric on xe-5/2/1.0 on cr2-eqiad, to move eqiad->pmtpa traffic to the lower latency link [production]
11:27 <pp-pdf2> restarted all services [production]
11:27 <pp-pdf1> restarted all services [production]
11:27 <pp-pdf3> restarted all services [production]
11:26 <pp-pdf3> updated mwlib.epub to 0.14.1 [production]
11:26 <pp-pdf2> updated mwlib.epub to 0.14.1 [production]
11:25 <pp-pdf1> updated mwlib.epub to 0.14.1 [production]
10:50 <mark> Setup semi-sync snapmirror from nas1-a:home_pmtpa to nas1001-a:home_pmtpa [production]
10:29 <pp-pdf3> restarted all services [production]
10:29 <pp-pdf1> restarted all services [production]
10:29 <pp-pdf2> restarted all services [production]
10:29 <pp-pdf1> update mwlib.epub to 0.14.0 [production]
10:28 <pp-pdf2> update mwlib.epub to 0.14.0 [production]
10:28 <pp-pdf3> update mwlib.epub to 0.14.0 [production]
10:28 <pp-pdf2> update mwlib.rl to 0.14.0 [production]
10:28 <pp-pdf1> update mwlib.rl to 0.14.0 [production]
10:28 <pp-pdf3> update mwlib.rl to 0.14.0 [production]
10:28 <pp-pdf2> update mwlib to 0.14.0 [production]
10:28 <pp-pdf3> update mwlib to 0.14.0 [production]
10:27 <pp-pdf1> update mwlib to 0.14.0 [production]
03:57 <mutante> nagios back up after adding missing monitor groups misc_pmtpa in appserver role (srv194) [production]
02:56 <K4-713> synchronized payments cluster to 37e31eddf5a4 [production]
02:28 <LocalisationUpdate> completed (1.20wmf9) at Thu Aug 16 02:28:33 UTC 2012 [production]
02:12 <Ryan_Lane> pushed in large puppet change for ldap, openstack, gerrit and ldap pdns to make it more modular and to add support for eqiad region [production]
02:10 <Ryan_Lane> fixed arecord issues with labsconsole by adding an exception handling live hack for the jobs [production]
01:21 <tstarling> synchronized wmf-config/InitialiseSettings.php 'remove wikimedia philippines event exemption' [production]
00:41 <mutante> installing package upgrades on singer [production]
00:36 <aaron> synchronized php-1.20wmf9/includes/filerepo/backend/FileBackendMultiWrite.php 'deployed c51a9a288b6dd5c0023a77f324c04707b23501c6' [production]
00:24 <binasher> running "mwscript purgeParserCache.php --wiki=$db --age=1814400" instead [production]
00:19 <binasher> running mwscript purgeParserCache.php --wiki=enwiki --age=1209600 [production]
00:02 <RoanKattouw> find /export/upload/wik*/*/{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,archive,math,temp,timeline} ! -user apache -exec /root/fixownership2 \\{\\} \\; where fixownership2 = chown -h apache $1 [production]
00:02 <RoanKattouw> Restarting the find-chown, this time with -h so symlinks are handled correctly (for some reason there are a bunch of broken symlinks with weird characters out there...) [production]
2012-08-15 §
23:52 <binasher> powercycled srv259 [production]
23:44 <mutante> brought nagios back up by removing "misc_pmtpa" host and servicegroups from srv194 manually [production]
23:35 <mutante> restarted pdns-recursor on ns0 [production]
23:11 <mutante> authdns-update, moving zirconium to public IP and removing selenium [production]
22:33 <reedy> synchronized php-1.20wmf9/extensions/OpenSearchXml/ [production]
21:19 <RoanKattouw> find /export/upload/wik*/*/{0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,archive,math,temp,timeline} ! -user apache -exec chown apache \\{\\} \\; [production]
21:17 <RoanKattouw> chown on ms7 has finished, running another find to find remaining bad files [production]