2101-2150 of 10000 results (74ms)
2018-10-29 §
10:22 <gehel> switch wdqs1003 and wdqs1006 completed, wdqs1003 still depooled to catch up on update lag - T207947 [production]
10:21 <hashar> Restore ti l10n files on deploy1001:/srv/mediawiki-staging/php-1.33.0-wmf.1/cache/l10n/upstream # T208196 [production]
10:15 <gilles@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T187299 Double sampling of performance perception survey on ruwiki (duration: 00m 47s) [production]
10:12 <hashar> Moving l10n_cache-ti.cdb files on deploy1001: sudo -u l10nupdate mv l10n_cache-ti.cdb.json l10n_cache-ti.cdb.json-back-T208196 && sudo -u l10nupdate mv l10n_cache-ti.cdb.MD5 l10n_cache-ti.cdb.MD5-back-T208196 # T208196 [production]
10:10 <addshore@deploy1001> Synchronized wmf-config: final sync (duration: 00m 47s) [production]
10:09 <addshore@deploy1001> Synchronized wmf-config/Wikibase-production.php: PT3/3 [[gerrit:470161]] Wikibase, Create and use wmgWikibaseRepoStatementSections (duration: 00m 47s) [production]
10:08 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: PT2/3 [[gerrit:470161]] Wikibase, Create and use wmgWikibaseRepoStatementSections (duration: 00m 47s) [production]
10:07 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: PT1/3 [[gerrit:470161]] Wikibase, Create and use wmgWikibaseRepoStatementSections (duration: 00m 50s) [production]
10:04 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: [[gerrit:470160]] Wikibase.php, move a bunch of config into clean area NOOP (duration: 00m 47s) [production]
10:03 <gehel> starting to switch wdqs1003 and wdqs1006 - T207947 [production]
10:02 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: [[gerrit:470159]] PT2/2 Wikibase, add IS.php setting for each possible extension (duration: 00m 47s) [production]
10:01 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:470159]] PT1/2 Wikibase, add IS.php setting for each possible extension (duration: 00m 47s) [production]
09:57 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:470158]] Wikibase, Remove unused wmgUseWikibaseQualityExternalValidation (duration: 00m 47s) [production]
09:55 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: PT2/2 [[gerrit:470156]] Wikibase, remove unused wmgWikibaseClientSettings (duration: 00m 47s) [production]
09:54 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: PT1/2 [[gerrit:470156]] Wikibase, remove unused wmgWikibaseClientSettings (duration: 00m 47s) [production]
09:51 <addshore@deploy1001> Synchronized wmf-config/Wikibase-production.php: pt3/3 [[gerrit:470155]] Wikibase, create and use wmgWikibaseClientInjectRecentChanges (duration: 00m 46s) [production]
09:50 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: pt2/3 [[gerrit:470155]] Wikibase, create and use wmgWikibaseClientInjectRecentChanges (duration: 00m 47s) [production]
09:49 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: pt1/3 [[gerrit:470155]] Wikibase, create and use wmgWikibaseClientInjectRecentChanges (duration: 00m 47s) [production]
09:45 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: [[gerrit:470154]] PT2/2 Wikibase, put all wgNamespaceAliases in IS.php (duration: 00m 46s) [production]
09:44 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:470154]] PT1/2 Wikibase, put all wgNamespaceAliases in IS.php (duration: 00m 47s) [production]
09:43 <addshore@deploy1001> sync-file aborted: [[gerrit:470154]] PT1/2 (duration: 00m 00s) [production]
09:41 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: [[gerrit:470153]] PT2/2 Wikibase, define $wgExtraNamespaces in IS.php (duration: 00m 47s) [production]
09:40 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:470153]] PT1/2 Wikibase, define $wgExtraNamespaces in IS.php (duration: 00m 46s) [production]
09:37 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: [[gerrit:470152]] Wikibase, kill $wmgWBSharedSettings (duration: 00m 47s) [production]
09:35 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: PT2/2 [[gerrit:470151]] Wikibase, move wmgWBSiteLinkGroups to IS.php (duration: 00m 46s) [production]
09:34 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: PT1/2 [[gerrit:470151]] Wikibase, move wmgWBSiteLinkGroups to IS.php (duration: 00m 47s) [production]
09:30 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: PT2/2 [[gerrit:470150]] Wikibase, Split specialSiteLinkGroups and manage from IS.php (duration: 00m 46s) [production]
09:30 <ema> resume cache hosts rolling reboots for kernel/microcode updates T203011 [production]
09:29 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: PT1/2 [[gerrit:470150]] Wikibase, Split specialSiteLinkGroups and manage from IS.php (duration: 00m 47s) [production]
09:19 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: PT2/2 [[gerrit:470339]] Introduce wmgWikibaseMaxSerializedEntitySize (duration: 00m 46s) [production]
09:17 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: PT1/2 [[gerrit:470339]] Introduce wmgWikibaseMaxSerializedEntitySize (duration: 00m 47s) [production]
09:07 <addshore@deploy1001> Finished scap: sync with no changes (duration: 14m 39s) [production]
08:56 <elukey> restart yarn on an-master100[1,2] to pick up new zookeeper timeout settings (10s -> 20s) - T206943 [production]
08:52 <addshore@deploy1001> Started scap: sync with no changes [production]
08:50 <addshore@deploy1001> sync aborted: (no justification provided) (duration: 00m 03s) [production]
08:50 <addshore@deploy1001> Started scap: (no justification provided) [production]
08:49 <addshore@deploy1001> sync-l10n aborted: (no justification provided) (duration: 01m 19s) [production]
08:42 <gilles@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T208088 Enable performance perception survey shuffling (duration: 00m 47s) [production]
08:38 <gilles@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/QuickSurveys: T208088 Add ability to shuffle answers display order (duration: 01m 51s) [production]
08:35 <gilles@deploy1001> sync aborted: T208088 Enable performance QuickSurvey shuffling (duration: 00m 00s) [production]
08:35 <gilles@deploy1001> Started scap: T208088 Enable performance QuickSurvey shuffling [production]
08:35 <gilles@deploy1001> sync aborted: T208088 Enable performance QuickSurvey shuffling (duration: 06m 30s) [production]
08:29 <gilles@deploy1001> Started scap: T208088 Enable performance QuickSurvey shuffling [production]
08:07 <godog> reformat ms-be1042 xfs filesystems - T199198 [production]
08:00 <gilles> Deploying time-sensitive backport to QuickSurveys [production]
02:08 <onimisionipe> repooling wdqs1003. It has caught up with others [production]
2018-10-28 §
23:36 <krinkle@deploy1001> Synchronized php-1.33.0-wmf.1/extensions/Graph: T184128 - I02da92de33 (duration: 00m 58s) [production]
19:17 <onimisionipe> depooling wdqs1003 to catch up on lag [production]
17:30 <elukey> restart yarn resource manager on an-master1002 to force failover to an-master1001 - T206943 [production]
16:36 <onimisionipe> repooling wdqs1003 - it didn't really catch up with others, but lag time on others are beginning to up. [production]