551-600 of 10000 results (15ms)
2013-08-19 §
19:03 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: special, closed, wikimedia, wikinews, wikivoyage to 1.22wmf13 [production]
18:05 <csteipp> Started syncing Wikimedia installation... : [production]
17:56 <paravoid> rebooting rubidium, mexia, eeden -- kernel upgrade [production]
17:51 <paravoid> apt: new geoip-database package version, from 2 years later [production]
17:42 <csteipp> synchronized php-1.22wmf12/extensions/CentralAuth 'revert ca update' [production]
17:35 <csteipp> synchronized php-1.22wmf12/extensions/CentralAuth 'Updating wmf12 CentralAuth to master for OAuth' [production]
17:12 <cmjohnson1> removing sq41 from dsh node groups and pybal [production]
16:33 <csteipp> synchronized php-1.22wmf13/extensions/CentralAuth 'Update CentralAuth to master for OAuth hooks' [production]
14:44 <paravoid> authdns-update: remove trailing dot from ulsfo reverse zone names [production]
13:16 <springle> temporarily running pt-kill on s5 slaves for slow wikidata queries [production]
12:49 <paravoid> hopefully final swiftrepl (swift->ceph thumbs) run on ms-fe1002 [production]
12:18 <tstarling> synchronized php-1.22wmf12/includes/GlobalFunctions.php 'revert temp hack, superseded' [production]
12:17 <faidon> synchronized wmf-config/filebackend.php 'ceph as a filebackend' [production]
11:58 <tstarling> synchronized wmf-config/CommonSettings.php 'changing session cookie name due to bug 53032' [production]
07:55 <tstarling> synchronized php-1.22wmf12/includes/GlobalFunctions.php 'hack for session bug' [production]
07:21 <paravoid> switching text-varnish pybal group back to squids [production]
07:20 <Tim> destroyed leaked session with ID 796c2b... [production]
05:49 <paravoid> authdns-update: add ADSP (dkim=unknown), DKIM policy record for lists (o=~) [production]
04:18 <paravoid> authdns-update: DKIM & DMARC (both no-op, for now) [production]
03:00 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Aug 19 03:00:26 UTC 2013 [production]
02:41 <LocalisationUpdate> completed (1.22wmf13) at Mon Aug 19 02:41:19 UTC 2013 [production]
02:15 <LocalisationUpdate> completed (1.22wmf12) at Mon Aug 19 02:15:45 UTC 2013 [production]
2013-08-18 §
18:54 <Elsie> rotated server admin log wikitech page; testing logging [production]
12:26 <paravoid> authdns-update: add AAAA for sodium [production]
11:43 <mark> Moved eqiad text-varnish traffic back to Squid temporarily [production]
10:14 <springle> resuming slave db52 after bug 52853 investigation [production]
07:18 <MaxSem> Killed a few hung queries on db1026 that were causing massive lag [production]
02:23 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Aug 18 02:23:36 UTC 2013 [production]
02:13 <LocalisationUpdate> completed (1.22wmf13) at Sun Aug 18 02:13:27 UTC 2013 [production]
02:06 <LocalisationUpdate> completed (1.22wmf12) at Sun Aug 18 02:05:50 UTC 2013 [production]
2013-08-17 §
15:02 <springle> synchronized wmf-config/db-eqiad.php 'returning db1009 to the pool after sync' [production]
02:40 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Aug 17 02:40:08 UTC 2013 [production]
02:20 <LocalisationUpdate> completed (1.22wmf13) at Sat Aug 17 02:19:58 UTC 2013 [production]
02:15 <LocalisationUpdate> completed (1.22wmf12) at Sat Aug 17 02:14:54 UTC 2013 [production]
2013-08-16 §
21:57 <demon> synchronized php-1.22wmf12/extensions/CirrusSearch 'No actual changes, was just a change + revert' [production]
21:46 <ori-l> Timeouts on last two sync-dirs: srv281, mw1089, mw1173 [production]
21:46 <olivneh> synchronized php-1.22wmf13/extensions/CoreEvents 'Updating CoreEvents to master for I08f66861b (2/2)' [production]
21:45 <olivneh> synchronized php-1.22wmf12/extensions/CoreEvents 'Updating CoreEvents to master for I08f66861b (1/2)' [production]
21:05 <reedy> synchronized docroot and w [production]
20:45 <reedy> synchronized docroot and w [production]
19:36 <cmjohnson1> shutting down brewster yet again to replace the disk [production]
17:30 <demon> synchronized php-1.22wmf13/extensions/CirrusSearch/CirrusSearch.body.php [production]
16:51 <cmjohnson1> brewster going down again [production]
16:21 <cmjohnson1> shutting down brewster to replace bad disk [production]
16:21 <bblack> merged vhtcpd ganglia stuff and the VCL for PURGE hash_ignore_busy into puppet [production]
15:09 <reedy> synchronized wmf-config/ 'touch' [production]
15:05 <reedy> synchronized database lists files: Ad tyvwiki to visualeditor.dblist [production]
09:36 <paravoid> rolling reboot of ceph ms-be nodes [production]
08:09 <apergos> sq41 hardware issues, see rt 5615 [production]
05:44 <paravoid> downgraded puppetmaster to previous release, new minor update had significantly increased CPU requirements and DoSed stafford [production]