51-100 of 10000 results (16ms)
2013-05-02 ยง
19:51 <Jeff_Green> authdns-update to move db1025 to frack.eqiad.wmnet [production]
19:46 <reedy> synchronized wmf-config/interwiki.cdb 'Updating interwiki cache' [production]
19:39 <ottomata> rebooting oxygen [production]
19:32 <ottomata> re-enabling puppet on cp1031, it was administratively disabled. running puppet there. [production]
19:23 <RobH> professor was already powered down (why?) starting it back up now [production]
19:22 <ottomata> all webrequest udp2log loggers (squid and varnish) now send to gadolinium for socat unicast -> multicast relay [production]
19:22 <RobH> rebooting professor [production]
19:16 <Jeff_Green> moving db1025 into frack-fundraising1-c-eqiad [production]
19:13 <kaldari> synchronized php-1.22wmf2/extensions/Echo 'sync Echo ext for en.wiki' [production]
19:13 <ottomata> deployed squid frontend.conf.php changes to remove locke and send logs directly to gadolinium for multicast relay [production]
19:11 <kaldari> synchronized php-1.22wmf3/extensions/Echo 'sync Echo ext' [production]
19:08 <kaldari> synchronized php-1.22wmf3/extensions/Echo 'sync Echo ext' [production]
18:51 <ottomata> removed varnishcsa-locke instance from varnish hosts: (dsh -c -g varnishncsa-all 'test -f /etc/init.d/varnishncsa-locke && service varnishncsa-locke stop && update-rc.d -f varnishncsa-locke remove && rm -v /etc/init.d/varnishncsa-locke') [production]
18:46 <reedy> synchronized wmf-config/interwiki.cdb 'Updating interwiki cache' [production]
18:40 <ottomata> varnishncsa now sends traffic to gadolinium instead of oxygen for multicast relay [production]
18:29 <Krinkle> Installed Monitoring plugin from Jenkins control panel <https://wiki.jenkins-ci.org/display/JENKINS/Monitoring> [production]
17:59 <Krinkle> Jenkins restart complete. No visible improvement. Jenkins is still idling most of the time while Zuul is still halted by an unknown factor on spawning jobs. [production]
17:56 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: [production]
17:53 <reedy> synchronized wmf-config/InitialiseSettings.php [production]
17:47 <reedy> synchronized wmf-config/ [production]
17:37 <Krinkle> Jenkins keeps clogging up. Starting an emergency restart. [production]
16:59 <Krinkle> Jenkins is nearing 100% CPU on gallium, what is Jenkins doing? [production]
16:59 <Krinkle> Zuul is somehow having trouble kicking off Jenkins jobs (less than 1 event processed per minute). Jenkins shows that 10/10 executors are idle. Investigating... [production]
16:52 <nikerabbit> synchronized php-1.22wmf3/extensions/UniversalLanguageSelector/ 'ULS to master' [production]
15:12 <hashar> Jenkins made Jenkins to instantly restart (was {{bug|47120}}) ) by deleting the downstream-buildview plugin. [production]
15:11 <hashar> restarted Zuul [production]
14:11 <hashar> Upgraded Jenkins from 1.480.3 to LTS 1.509.1. Restarted it. [production]
14:09 <hashar> shutting down Zuul [production]
14:08 <hashar> upgrading Jenkins (unscheduled maintenance). [production]
13:01 <maxsem> synchronized wmf-config/mobile.php 'https://gerrit.wikimedia.org/r/#/c/61941/' [production]
12:58 <mark> Migrated amslvs3 and amslvs4 PyBal BGP peerings from csw2-esams to cr2-knams [production]
09:54 <maxsem> synchronized php-1.22wmf2/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php 'https://gerrit.wikimedia.org/r/#/c/61926/' [production]
09:53 <maxsem> synchronized php-1.22wmf3/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php 'https://gerrit.wikimedia.org/r/#/c/61926/' [production]
09:50 <maxsem> synchronized php-1.22wmf3/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php 'https://gerrit.wikimedia.org/r/#/c/61926/' [production]
09:39 <maxsem> synchronized php-1.22wmf3/extensions/GeoData/GeoData.body.php 'https://gerrit.wikimedia.org/r/#/c/61962/' [production]
06:03 <apergos> powercycle snapshot1001, upgrade to precise [production]
05:45 <apergos> powercycle snapshot1004, upgrade to precise [production]
05:27 <apergos> snapshot1003 powercucle, upgrading to precise [production]
05:27 <Tim> on professor: manually started carbon-cache.py [production]
05:14 <Tim> professor is down, no response on serial console, rebooting [production]
05:11 <tstarling> synchronized wmf-config/jobqueue-pmtpa.php [production]
05:07 <tstarling> synchronized wmf-config/jobqueue-eqiad.php [production]
04:46 <Tim> on rdb1001-1002, set requirepass and masterauth in soft state [production]
04:44 <Tim> on mc1-16 and mc1001-1016, setting requirepass and masterauth to the new password in soft state [production]
04:39 <LocalisationUpdate> ResourceLoader cache refresh completed at Thu May 2 04:39:48 UTC 2013 [production]
04:03 <tstarling> synchronized wmf-config/CommonSettings.php [production]
03:59 <tstarling> synchronized wmf-config/CommonSettings.php [production]
03:56 <tstarling> synchronized wmf-config/jobqueue-eqiad.php [production]
03:46 <tstarling> synchronized php-1.22wmf2/extensions/GettingStarted [production]
03:39 <tstarling> synchronized php-1.22wmf3/extensions/GettingStarted [production]