|
2013-12-19
§
|
| 09:50 |
<apergos> |
powercycled cp1065, unresponsive, no messages on mgmt console |
[production] |
| 03:49 |
<Krinkle> |
Reloading Zuul to deploy I5b759c388db5e5a |
[production] |
| 03:21 |
<springle> |
starting schema changes for gerrit 101170 page_links_updated |
[production] |
| 02:58 |
<springle> |
synchronized database lists files: |
[production] |
| 02:53 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Dec 19 02:53:46 UTC 2013 |
[production] |
| 02:31 |
<LocalisationUpdate> |
completed (1.23wmf6) at Thu Dec 19 02:31:31 UTC 2013 |
[production] |
| 02:16 |
<LocalisationUpdate> |
completed (1.23wmf7) at Thu Dec 19 02:16:45 UTC 2013 |
[production] |
| 00:10 |
<subbu> |
updated parsoid config to add support for tyv and min wikipedias |
[production] |
| 00:09 |
<mutante> |
doing parsoid restart via salt, with the 10% option |
[production] |
|
2013-12-18
§
|
| 23:44 |
<Krinkle> |
Zuul pipeline for gate-and-submit fixed (the revert was someone undone after the reload, deployed the same git commit again) |
[production] |
| 23:36 |
<Krinkle> |
Reloading Zuul |
[production] |
| 23:34 |
<Krinkle> |
Zuul is once again not processing any CR+2 events |
[production] |
| 23:03 |
<ottomata> |
kafka brokers on an21 and an22 are now running with cleaned up topics |
[production] |
| 23:02 |
<ori-l> |
zirconium: service apache2 reload for wikimania scholarships app |
[production] |
| 22:38 |
<Krinkle> |
Reloading Zuul to deploy Ic8cf602a0aa72 |
[production] |
| 22:32 |
<Krinkle> |
Zuul seems to be unresponsive to gate-and-submit events for mediawiki/* projects |
[production] |
| 21:45 |
<ottomata> |
stopping kafka in order to cleanup old topic 'webrequest-mobile'. Usually we won't have to do cleanups like this, but since this topic is new and no one is using it yes, I'd rather clean up now rather than never.' |
[production] |
| 20:39 |
<awight> |
update crm from 7deaa1e1a6354de545fedb0e90b85f117e7df0bd to ce73eaa0ea6019f76c63d0cca097bf828eb933e2 |
[production] |
| 20:25 |
<reedy> |
updated /a/common to {{Gerrit|I33c68abb1}}: Have sysops add and remove users from the 'gwtoolset' group |
[production] |
| 20:12 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 19:06 |
<Reedy> |
Created EducationProgram tables on fawiki |
[production] |
| 19:05 |
<reedy> |
synchronized php-1.23wmf7/extensions/GWToolset/ |
[production] |
| 19:04 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 18:20 |
<otto> |
synchronized wmf-config 'Syncing wmf-config for Cirrus config updates' |
[production] |
| 18:19 |
<otto> |
synchronized cirrus.dblist 'Cirrus config update, adding new wikis' |
[production] |
| 17:52 |
<otto> |
synchronized php-1.23wmf6/extensions/CirrusSearch 'updating php-1.23wmf6/extensions/CirrusSearch to master' |
[production] |
| 17:51 |
<otto> |
synchronized php-1.23wmf6/extensions/CirrusSearch 'updating php-1.23wmf6/extensions/CirrusSearch to master' |
[production] |
| 17:46 |
<hashar> |
Zuul upgrade working as expected as far as I monitored it. Mail sent to wikitech-l and eng list. |
[production] |
| 17:27 |
<otto> |
synchronized php-1.23wmf7/extensions/CirrusSearch 'updating php-1.23wmf7/extensions/CirrusSearch to master' |
[production] |
| 17:23 |
<hashar> |
Zuul : speedy deploy of a Wikimedia hack to prevent Zuul from updating build description. Seems to slowing down everything. Hack is {{gerrit|102465}}. Deployed using https://www.mediawiki.org/wiki/Continuous_integration/Zuul#upgrading |
[production] |
| 16:46 |
<ottomata> |
updated varnishkafka in apt to 1.0.0-2 and installed on mobile varnishes (this includes fix for logrotate postrotate command) |
[production] |
| 15:40 |
<paravoid> |
powercycling cp1065 again, kernel BUG, soft lockup |
[production] |
| 15:29 |
<paravoid> |
powercycling cp1065, kmem_alloc deadlock |
[production] |
| 15:01 |
<hashar> |
Gerrit back :-) |
[production] |
| 15:00 |
<hashar> |
Gerrit dead :( |
[production] |
| 13:51 |
<paravoid> |
setting up new transit peering in eqiad |
[production] |
| 10:00 |
<ori-l> |
disabled puppet on tungsten to test mwprof changes; re-enabling now |
[production] |
| 07:37 |
<springle> |
online schema changes on masters with sql_log_bin=0, indexes only, gerrit 85508 |
[production] |
| 05:32 |
<ori-l> |
DNS update: scholarships.wikimedia.org CNAME misc-web-lb.eqiad |
[production] |
| 03:09 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Dec 18 03:09:14 UTC 2013 |
[production] |
| 02:47 |
<LocalisationUpdate> |
completed (1.23wmf6) at Wed Dec 18 02:47:41 UTC 2013 |
[production] |
| 02:25 |
<LocalisationUpdate> |
completed (1.23wmf7) at Wed Dec 18 02:25:17 UTC 2013 |
[production] |
| 01:52 |
<reedy> |
synchronized wmf-config/ 'touch' |
[production] |
| 01:46 |
<reedy> |
synchronized wmf-config/ 'Ensuring consistency...' |
[production] |
| 00:20 |
<mflaschen> |
synchronized wmf-config 'Deploy Draft namespace to English Wikipedia. Remove old Labs overrides that were used for testing this.' |
[production] |
| 00:12 |
<springle> |
ongoing schema changes on slaves, indexing only, logging gerrit 85508 |
[production] |
| 00:05 |
<Krinkle> |
Reloading zuul to deploy I6df1f4acccd7a9 |
[production] |