|
2019-01-04
§
|
| 23:07 |
<mutante> |
scandium apt-get remove nodejs nodes-legacy ; puppet agent -tv - after merging gerrit:482150 this fixed "you have held broken packages" issue, now we are at a puppet dependecy cycle with apt::pin T201366 |
[production] |
| 21:37 |
<bd808> |
Truncated /data/project/.system/accounting after archiving ~30 days of history |
[tools] |
| 18:17 |
<halfak> |
deploying ores-wmflabs-deploy:f3dbd2 |
[ores] |
| 18:13 |
<halfak> |
staging ores-wmflabs-deploy:f3dbd2 |
[ores] |
| 16:06 |
<halfak> |
ran "sudo service uwsgi-ores restart" on ores-staging-01 |
[ores] |
| 15:42 |
<bawolff@deploy1001> |
Synchronized private/PrivateSettings.php: T212667 - More aggressive anti-spam measures for account creation on kowiki (duration: 00m 48s) |
[production] |
| 15:39 |
<halfak> |
staging ores-wmflabs-deploy:7a3bdf |
[ores] |
| 15:10 |
<Hauskatze> |
restarted cvnbot3 on #cvn-wp-es |
[cvn] |
| 14:08 |
<moritzm> |
rebooting etcd1001-1003 to pick up SSBD-enabled qemu |
[production] |
| 14:05 |
<arturo> |
T212302 creating openstack-puppetmaster-01 and cloudvps-upgrade-test VM instances |
[openstack] |
| 13:52 |
<moritzm> |
rebooting etcd1004-1006 to pick up SSBD-enabled qemu |
[production] |
| 13:33 |
<moritzm> |
rebooting kubernetes staging etcd hosts to pick up SSBD-enabled qemu |
[production] |
| 13:11 |
<moritzm> |
rebooting kubernetes staging master to pick up SSBD-enabled qemu |
[production] |
| 12:57 |
<moritzm> |
rebooting kubernetes staging workers for kernel security update |
[production] |
| 11:58 |
<moritzm> |
installing libsndfile security updates |
[production] |
| 11:33 |
<moritzm> |
installing jasper security updates |
[production] |
| 11:31 |
<moritzm> |
installing libdatetime-timezone-perl updates for recent tz changes |
[production] |
| 11:11 |
<rxy> |
Flags +AV were set on shanmugamp7 in #cvn-mediawiki. |
[cvn] |
| 10:47 |
<arturo> |
T212898 reimaging cloudvirt1024 as stretch |
[production] |
| 10:46 |
<moritzm> |
rolling restart of swift proxies to pick up OpenSSL update |
[production] |
| 09:57 |
<jijiki> |
restarting thumbor services to pick up 481141 |
[production] |
| 09:50 |
<onimisionipe> |
restarting nginx on all wdqs hosts |
[production] |
| 09:42 |
<joal> |
Kill banner test kafka-druid ingestion job |
[analytics] |
| 09:40 |
<banyek> |
executing schema change on dbstore1002 - T85757 |
[production] |
| 09:13 |
<moritzm> |
restarting nginx on puppetdb hosts to pick up new OpenSSL |
[production] |
| 09:03 |
<banyek> |
executing schema change on db1116 - T85757 |
[production] |
| 08:44 |
<moritzm> |
restarting nginx on francium to pick up new OpenSSL |
[production] |
| 08:16 |
<elukey> |
restart eventlogging daemons on eventlog1002 to pick up openssl updates |
[analytics] |
| 08:16 |
<elukey> |
restart eventlogging daemons on eventlog1002 to pick up openssl updates |
[production] |
| 07:56 |
<moritzm> |
installing OpenSSL security updates |
[production] |
| 07:39 |
<elukey> |
decommission analytics1031/32 from the Hadoop analytics cluster |
[analytics] |
| 07:37 |
<elukey> |
manually stopped hdfs-balancer (cluster already balanced, only one host left with some blocks to get) to ease the decom of two more nodes |
[analytics] |
| 05:57 |
<Cam11598> |
restarted cvnbot19 |
[cvn] |
| 05:57 |
<Cam11598> |
9:56:34 PM <25B100+ CVNBot19> Last message was received on RCReader 5202.596163 seconds ago |
[cvn] |
| 00:07 |
<mutante> |
an-coord1001 - apt-get clean to free disk space, reacting to Icinga alert for running out of disk |
[production] |
|
2019-01-03
§
|
| 23:08 |
<volans> |
restarted pdfrender on scb1004 |
[production] |
| 22:29 |
<volans> |
restarted all slaves on dbstore1002 (relayed from banyek) |
[production] |
| 22:14 |
<banyek> |
stopping all slaves on dbstore1002 (NOT labsdb) |
[production] |
| 22:14 |
<banyek> |
stopping all slaves on labsdb1002 |
[production] |
| 21:03 |
<bd808> |
Enabled Puppet on tools-proxy-02 |
[tools] |
| 20:53 |
<bd808> |
Disabled Puppet on tools-proxy-02 |
[tools] |
| 20:51 |
<bd808> |
Enabled Puppet on tools-proxy-01 |
[tools] |
| 20:50 |
<reedy@deploy1001> |
Synchronized multiversion/MWMultiVersion.php: Fix error for testcommons (duration: 00m 44s) |
[production] |
| 20:49 |
<bd808> |
Disabled Puppet on tools-proxy-01 |
[tools] |
| 20:46 |
<reedy@deploy1001> |
Synchronized dblists/group0.dblist: Add testcommonswiki to group0 (duration: 00m 43s) |
[production] |
| 20:43 |
<reedy@deploy1001> |
Synchronized wmf-config/interwiki.php: Updating interwiki cache (duration: 02m 05s) |
[production] |
| 20:24 |
<reedy@deploy1001> |
Synchronized wmf-config/db-codfw.php: T197616 (duration: 00m 44s) |
[production] |
| 20:23 |
<reedy@deploy1001> |
Synchronized wmf-config/db-eqiad.php: T197616 (duration: 00m 44s) |
[production] |
| 20:15 |
<Hauskatze> |
gerrit: created https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/SparkPost |
[releng] |
| 20:13 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T197616 (duration: 00m 44s) |
[production] |