|
2017-04-20
§
|
| 09:38 |
<_joe_> |
live-hack redeployed, running scap pull on codfw jobrunners |
[production] |
| 09:34 |
<hashar@naos> |
Synchronized rpc/RunJobs.php: Revert "rpc: raise exception instead of die" - causes monitoring spam (duration: 01m 20s) |
[production] |
| 09:17 |
<_joe_> |
removed the live hack, running scap pull again on mw2154 |
[production] |
| 09:14 |
<_joe_> |
scap pull of live hack for T163418 on mw2154 |
[production] |
| 08:47 |
<_joe_> |
live-patching ./includes/jobqueue/jobs/RefreshLinksJob.php to drop all recursive jobs, T163418 |
[production] |
| 07:59 |
<jynus> |
shutting down db1080 for cloning and upgrade T163413 |
[production] |
| 07:54 |
<jynus@naos> |
Synchronized wmf-config/db-codfw.php: Add db2071, depooled (duration: 00m 53s) |
[production] |
| 07:53 |
<jynus@naos> |
Synchronized wmf-config/db-eqiad.php: Depool db1080 (duration: 01m 02s) |
[production] |
| 07:53 |
<marostegui> |
Deploy alter table enwiki.revision db1065 - https://phabricator.wikimedia.org/T132416 |
[production] |
| 07:31 |
<marostegui@naos> |
Synchronized wmf-config/db-eqiad.php: Depool db1065 - T132416 (duration: 02m 18s) |
[production] |
| 07:12 |
<marostegui> |
Deploy alter table on s4.image on eqiad master db1040 (this will create lag on eqiad - all hosts have been silenced) - https://phabricator.wikimedia.org/T73563 |
[production] |
| 06:39 |
<marostegui> |
Deploy alter table on s4.oldimage on eqiad master db1040 (this will create lag on eqiad - all hosts have been silenced) - T73563 |
[production] |
| 01:37 |
<mutante> |
mw2150 - restarted hhvm (had 'thread leakage' alert) |
[production] |
| 01:28 |
<mutante> |
ran puppet on all (16) Dell R320 via cumin to add CPU frequency check |
[production] |
| 00:37 |
<ejegg> |
updated CiviCRM from 90d679b171ee2791fbc30417265c09cd7140bfc7 to 51dbbad9f7822a7b3df730a2bd92ee3fb176b3ec |
[production] |
|
2017-04-19
§
|
| 23:58 |
<ejegg> |
updated payments-wiki from ccfbf98b475c135c1c1eb65ffe7dadaa6882461b to ee7d402216f6eae71cc85137741187ab096ead33 |
[production] |
| 22:37 |
<papaul> |
OS installation on db2071 |
[production] |
| 21:44 |
<ejegg> |
updated SmashPig from 17c56b016a12ff480358481c2b1de26c0acce83d to 200f63eedb05f5e6665d9837bba97e7e7237a41d |
[production] |
| 21:37 |
<krinkle@naos> |
Synchronized php-1.29.0-wmf.20/resources/src/startup.js: I34bbe8edf - Fix js fatal (duration: 01m 20s) |
[production] |
| 20:08 |
<ejegg> |
updated payments-wiki from 5398b23fd3c46c096222c986345bd77f5f813fb6 to ccfbf98b475c135c1c1eb65ffe7dadaa6882461b |
[production] |
| 19:22 |
<krinkle@naos> |
Synchronized php-1.29.0-wmf.20/resources/src/mediawiki/mediawiki.js: Ie50bdda229e48b (duration: 00m 58s) |
[production] |
| 19:20 |
<krinkle@naos> |
Synchronized php-1.29.0-wmf.20/extensions/WikimediaEvents/extension.json: T162604 (duration: 01m 20s) |
[production] |
| 19:17 |
<XenoRyet> |
Updated SmashPig from 3db064d258852b5c55184c991f183ecb3266f0c7 to 17c56b016a12ff480358481c2b1de26c0acce83d |
[production] |
| 18:58 |
<ejegg> |
rolled back payments-wiki to 5398b23fd3c46c096222c986345bd77f5f813fb6 |
[production] |
| 18:56 |
<ejegg> |
updated payments-wiki from 5398b23fd3c46c096222c986345bd77f5f813fb6 to 68e3ac6eea2820c5988b771e1f66e055a9aeeed9 |
[production] |
| 18:27 |
<ariel@naos> |
Finished deploy [dumps/dumps@ad621e6]: doc fixes thanks to awight (duration: 00m 04s) |
[production] |
| 18:26 |
<ariel@naos> |
Started deploy [dumps/dumps@ad621e6]: doc fixes thanks to awight |
[production] |
| 18:25 |
<ejegg> |
updated payments-wiki from 36f38f6c96887f328596a1b01ad6a03ef75a1b29 to 5398b23fd3c46c096222c986345bd77f5f813fb6 |
[production] |
| 18:19 |
<mobrovac> |
restbase stopping RB and disabling puppet on restbase1018 due to T163292 |
[production] |
| 18:18 |
<ariel@naos> |
Finished deploy [dumps/dumps@101f8a4]: page range fixes and standalone scripts (duration: 00m 18s) |
[production] |
| 18:18 |
<ariel@naos> |
Started deploy [dumps/dumps@101f8a4]: page range fixes and standalone scripts |
[production] |
| 17:27 |
<Amir1> |
mwscript extensions/ORES/maintenance/CleanDuplicateScores.php on all wikis with ORES review tool enabled (T163337) |
[production] |
| 17:26 |
<thcipriani@naos> |
Synchronized docroot/noc/index.html: test scap on naos.codfw.wmnet[[gerrit:348967|docroot/noc/index.html: trailing whitespace]] (duration: 02m 02s) |
[production] |
| 17:25 |
<mobrovac@naos> |
Started restart [restbase/deploy@1bfada4]: Restart to stop trying to connect to dead restbase1018 Cassandra instances - T163292 |
[production] |
| 17:08 |
<thcipriani@naos.codfw.wmnet> |
test |
[production] |
| 17:03 |
<filippo@naos> |
Finished deploy [prometheus/jmx_exporter@7327459]: test deploy from naos (duration: 00m 03s) |
[production] |
| 17:03 |
<filippo@naos> |
Started deploy [prometheus/jmx_exporter@7327459]: test deploy from naos |
[production] |
| 17:02 |
<godog> |
bounce tcpircbot on einsteinium to pick up changes |
[production] |
| 17:02 |
<_joe_> |
running manally enwiki refreshLinks jobs to catch up a bit |
[production] |
| 16:59 |
<papaul> |
power balancing on mw2215 |
[production] |
| 16:58 |
<Amir1> |
ladsgroup@naos:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=enwiki froze |
[production] |
| 16:49 |
<Amir1> |
ladsgroup@naos:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=enwiki (T163337) |
[production] |
| 16:33 |
<godog> |
deploy.fixurl on G@deployment_target:* after deployment server switchover |
[production] |
| 16:20 |
<gehel> |
disabling deprecation warning logs on elasticsearch eqiad - T163345 |
[production] |
| 16:19 |
<jynus> |
setting db2033 as read write |
[production] |
| 16:13 |
<godog> |
run puppet on naos.codfw.wmnet - new deployment server |
[production] |
| 16:09 |
<MediaWiki_> |
test |
[production] |
| 16:03 |
<gehel> |
disabling deprecation warning logs on elasticsearch codfw - T163345 |
[production] |
| 15:55 |
<User___> |
test |
[production] |
| 15:51 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: dc=codfw,cluster=elasticsearch,name=elastic2020.* |
[production] |