251-300 of 10000 results (13ms)
2014-10-08 §
00:41 <mutante> searchidx1001 - same, fixed duplicate salt-minion [production]
00:40 <mutante> osmium - salt-minion was running twice, stopped both, killed one, restarted properly [production]
00:38 <mutante> cp3016 - why you report failed puppet unlike everyone else but then it works [production]
00:34 <springle> long schema changes running from terbium. ok to kill osc_host.sh in emergency [production]
00:01 <ori> Synchronized php-1.25wmf2/extensions/WikimediaEvents: Update WikimediaEvents for If9cdde0f0 (duration: 00m 03s) [production]
00:01 <ori> Synchronized php-1.25wmf1/extensions/WikimediaEvents: Update WikimediaEvents for If9cdde0f0 (duration: 00m 04s) [production]
2014-10-07 §
23:29 <andrewbogott> restarting every shutoff VM on virt1005 [production]
23:20 <maxsem> Synchronized wmf-config/InitialiseSettings.php: SWAT: https://gerrit.wikimedia.org/r/165393 (duration: 00m 04s) [production]
22:54 <cscott> updated OCG to version c778ea8b898f8ad8c2b7ad9de78a75469e7ed061 [production]
22:50 <mutante> db68,tarin - revoke the last remaining pmtpa certs [production]
22:48 <ori> Synchronized php-1.25wmf1/extensions/WikimediaEvents: Update WikimediaEvents for Ied71b5032: Groundwork for HHVM productivity analysis (duration: 00m 04s) [production]
22:47 <mutante> db60,db69-74,es4,es7,es10 - remove from icinga monitoring, puppet certs, salt keys [production]
22:42 <ori> Synchronized php-1.25wmf2/extensions/WikimediaEvents: Update WikimediaEvents for Ied71b5032: Groundwork for HHVM productivity analysis (duration: 00m 04s) [production]
22:40 <mutante> fenari - revoked puppet cert, rm salt key, rm from icinga ... [production]
22:37 <andrewbogott> cycling power on virt1005 -- unresponsive [production]
21:27 <mutante> mchenry - revoke puppet cert, clean storedconfigs/rm from icinga [production]
21:04 <mutante> dobson - revoke puppet cert, delete from storedconfigs/icinga, deleted from dsh [production]
20:56 <K4-713> altered worldpay account settings for France on payments [production]
20:48 <mutante> mexia - revoke salt,puppet,monitoring,storedconfigs [production]
20:27 <mutante> pdf2/pdf3 - revoked puppet certs, removed from DNS & icinga [production]
19:42 <mutante> temp. stopped icinga-wm [production]
19:41 <mutante> restarting apache on palladium - mod_passenger fail [production]
19:30 <reedy> Synchronized wmf-config/: (no message) (duration: 00m 23s) [production]
19:29 <reedy> Synchronized database lists: (no message) (duration: 00m 20s) [production]
19:20 <Reedy> Created EducationProgram tables on cawiki [production]
19:19 <reedy> Synchronized wmf-config/: (no message) (duration: 00m 26s) [production]
19:09 <^d> cleared old files from runs on gallium tmpfs, testing should recover now. [production]
18:45 <csteipp> deployed fix for bug 71749 [production]
18:43 <mutante> sanger - deleted salt key, revoked puppet cert, rm icinga stored config, already out of DNS - Killing sanger.wikimedia.org...done. [production]
18:42 <csteipp> Synchronized php-1.25wmf2/extensions/CentralAuth/CentralAuthPlugin.php: (no message) (duration: 00m 06s) [production]
18:34 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf2 [production]
18:25 <^d> jenkins tmpfs run out of space again, tests failing [production]
16:24 <reedy> Synchronized database lists: echo for fawikivoyage (duration: 00m 20s) [production]
16:22 <Reedy> Created echo tables on fawikivoyage on extension1 cluster [production]
15:00 <reedy> Synchronized docroot and w: (no message) (duration: 00m 14s) [production]
14:31 <reedy> Synchronized docroot and w: Fixup noc (duration: 00m 16s) [production]
10:53 <godog> restart commons swiftrepl from ms-fe1003 and non-commons from ms-fe1004 to avoid maxing out copper's nic [production]
10:09 <godog> start swiftrepl of commons originals eqiad -> codfw [production]
09:56 <godog> start swiftrepl of non-commons originals eqiad -> codfw [production]
08:55 <Krinkle> Building additional contint slaves in labs (integration-slave1004 with precise and integration-slave1009 with trusty) [production]
08:21 <Krinkle> Reload Zuul to deploy 5e905e7c9dde9f47482d [production]
06:02 <ori> Synchronized php-1.25wmf1/includes/objectcache/HashBagOStuff.php: I0b0b5f01: HashBagOStuff: use the value itself as the CAS token (duration: 00m 06s) [production]
06:02 <ori> Synchronized php-1.25wmf2/includes/objectcache/HashBagOStuff.php: I0b0b5f01: HashBagOStuff: use the value itself as the CAS token (duration: 00m 07s) [production]
03:28 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Oct 7 03:28:09 UTC 2014 (duration 28m 8s) [production]
02:26 <LocalisationUpdate> completed (1.25wmf2) at 2014-10-07 02:26:04+00:00 [production]
02:14 <LocalisationUpdate> completed (1.25wmf1) at 2014-10-07 02:14:21+00:00 [production]
01:16 <ori> Synchronized php-1.25wmf1/extensions/Wikidata: Ie92da71 / I44f1dce: Update Wikidata, fixes for serialization issues (duration: 00m 09s) [production]
01:15 <ori> Synchronized php-1.25wmf2/extensions/Wikidata: Ie92da71 / I44f1dce: Update Wikidata, fixes for serialization issues (duration: 00m 10s) [production]
00:34 <Tim> core dumps were enabled on mw1088, unexpectedly started gathering natural segfault traffic [production]
2014-10-06 §
23:49 <mutante> tarin, nfs-1 - revoked salt key,puppet cert, stored configs [production]