6351-6400 of 10000 results (51ms)
2014-10-22 §
13:38 <godog> catch-up swiftrepl sync eqiad -> codfw for commons containers [production]
09:41 <hashar> Zuul/Jenkins in a deadlock [production]
09:08 <hashar> Restarting Jenkins [production]
09:07 <hashar> Jenkins: upgrading gearman-plugin from 0.0.7-1-g3811bb8 to 0.1.0-1-gfa5f083 . Ie bring us to latest version + 1 commit [releng]
09:07 <hashar> Jenkins: upgrading gearman-plugin from 0.0.7-1-g3811bb8 to 0.1.0-1-gfa5f083 . Ie bring us to latest version + 1 commit [production]
03:54 <LocalisationUpdate> ResourceLoader cache refresh completed at Wed Oct 22 03:54:53 UTC 2014 (duration 54m 52s) [production]
02:30 <LocalisationUpdate> completed (1.25wmf4) at 2014-10-22 02:30:07+00:00 [production]
02:17 <LocalisationUpdate> completed (1.25wmf3) at 2014-10-22 02:17:49+00:00 [production]
2014-10-21 §
23:21 <maxsem> Synchronized php-1.25wmf4/includes/PrefixSearch.php: https://gerrit.wikimedia.org/r/#/c/167982/ (duration: 00m 03s) [production]
23:09 <maxsem> Synchronized php-1.25wmf3/extensions/MobileFrontend/: SWAT (duration: 00m 04s) [production]
23:08 <maxsem> Synchronized php-1.25wmf4/extensions/MobileFrontend/: SWAT (duration: 00m 04s) [production]
23:07 <maxsem> Synchronized php-1.25wmf4/extensions/CentralAuth/: SWAT (duration: 00m 04s) [production]
22:48 <mutante> radium - installed OS, signing puppet cert requests, initial run ... [production]
22:42 <legoktm> manually finished global rename for BonumTV --> Karypal which failed due to page move timeout [production]
22:13 <andrewbogott> deleted unused labs projects: versionview, feeds, datadog, fundraising-awight, simplewiki, mediawiki-custom-de, fundraising, sartoris, wikibits, incubator, wikiversity-sandbox, data4all [production]
21:33 <K4-7131> adjusting payments antifraud filters [production]
21:29 <ebernhardson> Synchronized php-1.25wmf4/extensions/LiquidThreads/api/ApiQueryLQTThreads.php: Bump LQT in 1.25wmf4 (duration: 00m 04s) [production]
21:10 <hashar> contint: refreshed slave-scripts 0b85d48..8c3f228 sqlite files will be cleared out after 20 minutes (instead of 60 minutes) {{bug|71128}} [releng]
20:51 <cscott> deployment-prep _joe_ promises to fix this properly tomorrow am [releng]
20:51 <cscott> deployment-prep turned off puppet on deployment-pdf01, manually fixed broken /etc/ocg/mw-ocg-service.js [releng]
20:50 <cscott> deployment-prep updated OCG to version 523c8123cd826c75240837c42aff6301032d8ff1 [releng]
20:41 <cscott> updated OCG to version 523c8123cd826c75240837c42aff6301032d8ff1 [production]
20:40 <hoo> Synchronized wmf-config/CommonSettings.php: Set extendwatchlist = 0 (duration: 00m 08s) [production]
20:37 <hashar> lanthanum /var/lib/jenkins-slave/tmpfs went full again. cleared up a bunch of files [production]
20:22 <_joe_> installing new hhvm packages on the depooled server mw1189, for debugging [production]
20:14 <AaronSchulz> Deployed deployment/jobrunner to d426235e10edc682b532e7b4f2b02bb9414661ba [production]
19:47 <hoo> Synchronized wmf-config/InitialiseSettings.php: Add import sources for orwikisource (duration: 00m 08s) [production]
19:12 <qchris> Added its-phabricator plugin (d425a5ded909ee73df53d5e6d91d28014d0be375) into gerrit [production]
18:40 <chasemp> puppet on gerrit which restarted service [production]
18:25 <reedy> Synchronized php-1.25wmf3/extensions/Collection/: (no message) (duration: 00m 13s) [production]
18:24 <reedy> Synchronized php-1.25wmf4/extensions/Collection/: (no message) (duration: 00m 14s) [production]
18:21 <reedy> Synchronized database lists: (no message) (duration: 00m 26s) [production]
18:20 <reedy> Synchronized wmf-config/: (no message) (duration: 00m 18s) [production]
18:08 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf4 [production]
16:54 <_joe_> stopping puppet on mw1114 in order to do some jemalloc debugging [production]
16:40 <K4-713> adjusted payments fraud filters for WP test [production]
14:45 <godog> start catch up swiftrepl on ms-fe1003 for 'notcommons' containers [production]
14:28 <ottomata> set vm.dirty_writeback_centisecs = 200 (was 500) on analytics1021 [production]
14:23 <springle> Synchronized wmf-config/db-eqiad.php: depool db1065 (duration: 00m 06s) [production]
11:56 <godog> silenced *-lb.ulsfo.wikimedia.org [production]
11:43 <godog> drained ulsfo via DNS, GTT link problems [production]
11:10 <springle> Synchronized wmf-config/db-eqiad.php: repool db1066, warm up (duration: 00m 06s) [production]
10:55 <hashar> deleted salt master key on deployment-elastic{06,07}, restarted salt-minion and reran puppet. It is now passing on both instances \\O/ [releng]
10:48 <hashar> rerunning puppet manually on deployment-elastic{06,07} [releng]
10:48 <hashar> beta: signing puppet cert for deployment-elastic{06,07}. On deployment-salt ran: puppet ca sign i-000006b6.eqiad.wmflabs; puppet ca sign i-000006b7.eqiad.wmflabs [releng]
09:29 <hashar> forget me deployment-logstash1 has a puppet agent error but it is simply because the agent is disabled "'debugging logstash config'" [releng]
09:28 <hashar> deployment-logstash1 disk full [releng]
09:09 <akosiaris> enabled icinga-wm again [production]
08:50 <akosiaris> temporarily killed icinga-wm [production]
04:21 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Oct 21 04:21:19 UTC 2014 (duration 21m 18s) [production]