7101-7150 of 10000 results (70ms)
2019-03-12 §
08:36 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1082 for schema change (duration: 00m 48s) [production]
08:34 <jynus> deploy core replica events to db1118 [production]
08:15 <ema> cp1099: ferm.service failed to resolve prometheus1003.eqiad.wmnet. ferm restarted T202966 [production]
07:18 <marostegui> Deploy schema change on db2052 (s5 codfw master), this will generate lag on codfw T71127 T51199 [production]
07:17 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1113 after schema change and upgrade (duration: 00m 49s) [production]
07:09 <marostegui> Upgrade mysql and kernel on db1113 [production]
06:40 <kart_> Finished manual run of unpublished ContentTranslation draft purge script (T217818) [production]
06:17 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1113 for schema change and upgrade (duration: 00m 50s) [production]
05:20 <rxy> Flags +AV were set on cohaf in #cvn-simplewikis. [cvn]
05:18 <rxy> Flags +AV were set on Tulsi in #cvn-sw. [cvn]
04:04 <kart_> Started manual run of unpublished ContentTranslation draft purge script (T217818) [production]
03:46 <bd808> Restarted uwsgi-invisible-unicorn on proxy-01 [project-proxy]
03:21 <bd808> Removed redis sets with no record in the backing database (T133554) [project-proxy]
03:00 <bd808> Fixed domains with trailing dot (T218064) [project-proxy]
02:40 <ejegg> updated payments-wiki from f1a89d7045 to 7a312e371a [production]
01:13 <bd808> Deleted dangling backend records in /etc/dynamicproxy-api/data.db (T218064) [project-proxy]
00:22 <bd808> Switched *.tiles.wmflabs.org proxies to point to http://172.16.5.154:80 (T217992) [maps]
2019-03-11 §
18:46 <Joan> Restarted CVNBot18 (Last message was received on RCReader 29613.939513 seconds ago). [cvn]
18:09 <gtirloni> restarted shinken to clear invalid alerts [shinken]
17:55 <addshore@deploy1001> Synchronized wmf-config/interwiki-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495723/ (duration: 00m 48s) [production]
17:47 <bd808> Deployed 6b6d5cc Explicitly cast numeric data to int [tools.sge-status]
17:43 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495721/ (duration: 00m 49s) [production]
17:23 <arturo> T215605 copy python-oath from jessie-wikimedia/thirdparty to stretch-wikimedia/thirdpary in reprepro [production]
17:02 <jbond@cumin1001> END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) [production]
17:02 <jbond@cumin1001> START - Cookbook sre.hosts.ipmi-password-reset [production]
16:31 <jbond@cumin1001> END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) [production]
16:31 <jbond@cumin1001> START - Cookbook sre.hosts.ipmi-password-reset [production]
16:20 <bd808> Updated to dbb5f60 Guard against malformed accounting lines [tools.grid-jobs]
15:59 <Krenair> previous !log was for deployment-prep [releng]
15:56 <Krenair> added MSantos to projectadmin, per chat in -infrastructure he'll use it to work on maps. user is a new foundation software engineer [releng]
15:53 <bd808> Manually started `service gridengine-master` on tools-sgegrid-master after reboot (T218038) [tools]
15:47 <bd808> Hard reboot of tools-sgegrid-master via Horizon UI (T218038) [tools]
15:42 <bd808> Rebooting tools-sgegrid-master (T218038) [tools]
15:26 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1097 (duration: 00m 48s) [production]
15:16 <mlitn@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Fix syntax for MediaInfo depicts config (beta only) (duration: 00m 49s) [production]
14:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1097 (duration: 00m 49s) [production]
14:49 <gtirloni> deleted tools-webgrid-lighttpd-1419 [tools]
14:43 <moritzm> upgrading mw canaries to PHP 7.2.16 [production]
14:32 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1097 (duration: 00m 48s) [production]
14:25 <hashar> contint1001: stopping zuul-merger (it is cpu or IO starving the server) [production]
14:21 <moritzm> upgrading mwdebug servers to PHP 7.2.16 [production]
14:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1097 (duration: 00m 47s) [production]
14:09 <moritzm> importing build of PHP 7.2.16 for component/php72 (T216712) [production]
13:58 <marostegui> Upgrade mysql on db1097 [production]
13:28 <arturo> disable active checks in icinga for labtestvirt200[12] (T218023) [production]
13:04 <moritzm> upgrading mwdebug2002 to php 7.2.16 [production]
12:23 <gtirloni> updated slapd to version 2.4.47 on serpens (T217280) [production]
12:05 <gtirloni> updating slapd on serpens/codfw to test possible fix for memory leaks [production]
10:48 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1097 for upgrade and schema change (duration: 00m 48s) [production]
10:44 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:495650| Bumping portals to master (T128546)]] (duration: 00m 49s) [production]