TLS Reports from microsoft.com are not parsed

[makuartur]

The TLS report from microsoft.com wasn't parsed and was moved to the invalid folder
From:
[email protected]

Subject:
Report Domain: <domain_name>
Submitter: microsoft.com
Report-ID: ^[0-9]{18}+<domain_name>

Body:
_This is an aggregate TLS report from microsoft.com

Microsoft respects your privacy. Please review our online Privacy Statement.

Microsoft Corporation
One Microsoft Way
Redmond, WA, USA 98052_

Attached filename:
microsoft.com!domain_name!1725840000!1725926399!report_id.json.gz

Content of the file:

{
  "organization-name": "Microsoft Corporation",
  "date-range": {
    "start-datetime": "2024-09-09T00:00:00Z",
    "end-datetime": "2024-09-09T23:59:59Z"
  },
  "contact-info": "[email protected]",
  "report-id": "report_id+domain_name",
  "policies": [
    {
      "policy": {
        "policy-type": "sts",
        "policy-string": [
          "version: STSv1",
          "mode: enforce",
          "mx: aspmx.l.google.com",
          "mx: alt1.aspmx.l.google.com",
          "mx: alt2.aspmx.l.google.com",
          "mx: alt3.aspmx.l.google.com",
          "mx: alt4.aspmx.l.google.com",
          "max_age: 604800"
        ],
        "policy-domain": "domain_name"
      },
      "summary": {
        "total-successful-session-count": 2,
        "total-failure-session-count": 0
      }
    }
  ]
}

parsedmarc version: 8.14.1

[seanthegeek]

Running the raw json file you provided through parse_report_file() parses the report correctly. So, the issue must be with the email itself. I don't have any email samples from Microsoft to test with. Can you provide one?

[makuartur]

Will this option suit you?
Delivered-To: [email protected]
Received: by 2002:a05:612c:1a52:b0:48e:c7f6:a1f with SMTP id hu18csp1070346vqb;
Thu, 12 Sep 2024 13:42:36 -0700 (PDT)
X-Google-Smtp-Source: AGHT+IH6Cy3MXLWC6ihmN3fU8IWcv+413i8hf9sdXzU8zeg2i0udUB4CdrelYSfEipFvr531u0Qy
X-Received: by 2002:a05:6a21:1190:b0:1cf:6baf:61c0 with SMTP id adf61e73a8af0-1cf764c29e6mr5901032637.44.1726173756013;
Thu, 12 Sep 2024 13:42:36 -0700 (PDT)
ARC-Seal: i=2; a=rsa-sha256; t=1726173756; cv=pass;
d=google.com; s=arc-20240605;
b=PdN1ae9sPwdb2g3VeZSUMKTpwv9yEvArk2scl+IFUs8RWqKJJ1myqmvaIMSdFQnvCY
UzP8xLIwYbdjhNTLGbWy+QIqnCpAd9+vrwPPnAhpvRKKtage4yM7ZHBSi/kbld4KRJNv
LagNakHURiLIec92qPs7EJqbtSvtqE33sVOo5+26RVSge8eJW3z/gtdFrg4HOx6qM9kv
hQ/EV/L9O/WdL+M6/q0ZuxonyNe2JSYr7wrj5U3iAL0eUTSV6VW/9tXbmbgil6zUigSX
5/d2ShvKqU5oGJzzBOdxbHMoOkVgHjgT3wQ3/mRjjekgHmr7FV4R7FcfGGBWLrnIRapv
cRxQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605;
h=message-id:mime-version:tls-report-submitter:tls-report-domain
:subject📅to:from:dkim-signature;
bh=hMoLhrNBXNEU8Zpp9QAId7yWfsZ80FBUxqB9dt0ycRg=;
fh=NlfXymrgzSwYC67628bBfGK4hKLRfJcdy8QuPYRp5f4=;
b=Iq68YMfs5wcf9cF18VW2zdHpWuY4Eiyx+fqSD+iDGJd1uPo4yZh1bk5IKd7+WYSIzK
v+D3cU9mC1TmF9q5HfIzY5tepDUNUUv1RRkk7J2H7htYowyrvCYIwN84xZQEZKAsrpMc
a2UQcjraXYbdY8XiHtNLhVODhJCcQsCqlCgl98HhLIdyaXgbPnsv1scps0Vz1uAi5351
Qa+G00IpehzRf12+ZxRrXqtlsY95Yc8gSaEHyX9W9swx+5CnM4p6HS9TcL+75iYVjY0Y
GDnUDcwu2mvj0lL7iWVqFYgPCX8uuDArqqMuyqOe47PUrTVQsHXsnlZHKFCkvrFKwsFZ
1mbg==;
dara=google.com
ARC-Authentication-Results: i=2; mx.google.com;
dkim=pass [email protected] header.s=selector2 header.b=MnrBBYSL;
arc=pass (i=1);
spf=pass (google.com: domain of [email protected] designates 2a01:111:f403:c000::1 as permitted sender) smtp.mailfrom=[email protected];
dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=microsoft.com
Return-Path: [email protected]
Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazlp170100001.outbound.protection.outlook.com. [2a01:111:f403:c000::1])
by mx.google.com with ESMTPS id 41be03b00d2f7-7db1fe308b6si3314850a12.780.2024.09.12.13.42.35
for [email protected]
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Thu, 12 Sep 2024 13:42:35 -0700 (PDT)
Received-SPF: pass (google.com: domain of [email protected] designates 2a01:111:f403:c000::1 as permitted sender) client-ip=2a01:111:f403:c000::1;
Authentication-Results: mx.google.com;
dkim=pass [email protected] header.s=selector2 header.b=MnrBBYSL;
arc=pass (i=1);
spf=pass (google.com: domain of [email protected] designates 2a01:111:f403:c000::1 as permitted sender) smtp.mailfrom=[email protected];
dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=microsoft.com
ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;
b=KNelPmz4Bk31LbzUSMsvaJnGO7k4W9t9E3B0qgwwab0Rs59A5s6lMokpa1WMnFZvjehmT9Ienm8kIG0z+wkLwWU2nGBBYPTcvtH7LelOkNPV2DYgw3F+J2meR3KqQ0yoy2zCKJjhCna1KI2sQd+7CeFZXNXMrjRGwVLmdPoZeuTSCKxzF8A1MOOjoGv3tc1KXTSvAAbKyxX2pXhhy+PkkJI7F6433+9gxGCFm4r9FISUR8ov+ky9/aoioG4Ju7vt3tBRs/Kw28j7FzYXUvaHOtLc6L1aXwl/2Ylg4Hwz19TpcwnfWs+tKHitEJgVsAO2m1c11Lyq63rqRpbkM6Hdzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector10001;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=hMoLhrNBXNEU8Zpp9QAId7yWfsZ80FBUxqB9dt0ycRg=;
b=xOAPZWjLxfM357qsInlVxF1tGQXUyJuEFKDYrG3LbHByjl81tN/GGn3lWbR0VJldJLsrkf4thku11X8vbt35tdAJMHibvXx+85FnYGPiPqOe8PSIcGCfhJYHhLBWW+ZjZTw2Q2ptwo2/IpVstVemeZ2tp1gYZh/XmtI4v3JBhl325q/Nd3Gd5u2trmCRyFKSolXO4oGvEh/fC1KA1VNO9WIXnOgd6JK2twX9qPsyECJvEE3uRwMAz9nAdZyS7sXT526vEevYu8HoHvptejFVSgo3QRuq4UihHPvWuEOa/sQWOJ1YUDxg67t46nCkKsB5mEPrHZWMEHynCfCRpHjfzQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none
action=none header.from=microsoft.com; dkim=none (message not signed);
arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=selector2;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=hMoLhrNBXNEU8Zpp9QAId7yWfsZ80FBUxqB9dt0ycRg=;
b=MnrBBYSL8p8lsu+H6coxDIm2W8mcubWuY095SpN/1hcPWMrQ0XfHUEfIxhxV2XZqCC2Dbmsmv9CVBlGcijye5Y0RboUfu6YeFPZr2X+et4gRKpZP4akxVgemXo9SOf22dmnePkIY6pPWDOyJxp8LEh8ySE7EJbsPNsmNlSQsDa4=
Received: from MN2PR00CA0008.namprd00.prod.outlook.com (2603:10b6:208:224::21)
by IA2PR21MB4323.namprd21.prod.outlook.com (2603:10b6:208:4b0::22) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.6; Thu, 12 Sep
2024 20:42:32 +0000
Received: from BL2NAM06FT015.Eop-nam06.prod.protection.outlook.com
(2603:10b6:208:224:cafe::af) by MN2PR00CA0008.outlook.office365.com
(2603:10b6:208:224::21) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7999.0 via Frontend
Transport; Thu, 12 Sep 2024 20:42:31 +0000
X-MS-Exchange-Authentication-Results: spf=none (sender IP is 172.179.146.41)
smtp.mailfrom=microsoft.com; dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=microsoft.com;
Received: from 104.47.53.36 (172.179.146.41) by
BL2NAM06FT015.mail.protection.outlook.com (10.152.107.16) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.7982.4 via Frontend Transport; Thu, 12 Sep 2024 20:42:31 +0000
From: [email protected]
To: "[email protected]" [email protected]
Date: Thu, 12 Sep 2024 20:42:31 +0000
Subject: Report Domain: fake.domain.name Submitter: microsoft.com Report-ID:
111111111111111111+fake.domain.name
TLS-Report-Domain: fake.domain.name
TLS-Report-Submitter: microsoft.com
MIME-Version: 1.0
Message-ID: [email protected]
Content-Type: multipart/report;
boundary="a737a3b9-5ea5-40e8-a971-a67ecbb2993e"; report-type=tlsrpt
Return-Path: [email protected]
X-MS-TrafficTypeDiagnostic:
BL2NAM06FT015:EE_FirstParty-TlsRpt-V3-System|IA2PR21MB4323:EE_FirstParty-TlsRpt-V3-System
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: ae339a39-51d3-4824-723f-08dcd36b6cb3
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|61400799027;
X-Microsoft-Antispam-Message-Info:
=?us-ascii?Q?X2r3WPqkzvTm/A5lM6ZYm9IR4lza9Y1O8dF0fsIkNYvqmwis6K/K9Nb5JQm0?=
=?us-ascii?Q?MNbSgbbreHa75V1ewsp3C+faU6o6HW+mi6Qk+brxHpHoNYP96fQvyRlG3liQ?=
=?us-ascii?Q?s+2xvt6S8ud9rDGXKHA7UQ/YfVQShsci7iKmgQ729XhBqq3YzaM3VkV/mdNF?=
=?us-ascii?Q?d2o/Pnl/C63Plyc118ZCxKRT8ezmnXY2UwHrDN5N8kwAIO63Tc+4TYDRiNKJ?=
=?us-ascii?Q?h9M4iPAiUI9nEXXyoZg0xlcYAVL7iiT2sJO534MLsxcD3Z/UudaVjhjBcs4u?=
=?us-ascii?Q?59FpMsXxJxDLQs9OMvwaH2smwQE3VFf9QL3HNc+iT9G/frViGaEZen2rNu0E?=
=?us-ascii?Q?X9uHQbqIj+X2Nk3TX1ytsRLfrueUPTgFSKEqEuF8qZErfzbd5qtCA/2CUHN2?=
=?us-ascii?Q?GlwLjoKErzWNOzkxwSS1dy/FquA4cCSpSvJWiJjSmIZJ4kfJqcNiwZb4AfZt?=
=?us-ascii?Q?g63Sol3MBTC6DPlr/PuXWHgktCwMsy0ElbGmFmc0I0f00ts7h2Jdjc9VCP2c?=
=?us-ascii?Q?+Qz/FazYXoeNqg/KC72u3I5uJkZtrmnFzjde3uD4MHYUdMnAgfhbbZZpmUPw?=
=?us-ascii?Q?QNVUZX3GHuJZcri4QkAANPaVhrbidsQflLpPTVBJE7BflmKCDzsUib6SW0Xi?=
=?us-ascii?Q?lk71c9/iATSlm/LdXpRuxDqzAndEoIm7dllqOSWDbqZQXP7f/eEnW0zBxWqB?=
=?us-ascii?Q?i1a+9sphIyiPSxcaMLcu4+aH9MpU4sX9FWsOJYoZBlF8mhUjC455pTzD55Dp?=
=?us-ascii?Q?uihDJ27e5is+BApIKnr0aXFpGH7aFD6IHPKUEV5BkK6PXfPgb/GMQhpHuG66?=
=?us-ascii?Q?fQ3c3nTOgNqpTbxr04FKFIXhRBDS3S8d5y4w9+cvrm6r0yY3M8rjn5DEVDH+?=
=?us-ascii?Q?AYiCH9vyTJL3Rsx0TaHrtoY37Aohv1etjpcVTt3WhzFnS4l6T8NpIr1X1Y4z?=
=?us-ascii?Q?bfTqNqzAuQi76RGRyGv4TBVAY90qZ7pzKAEBfI7HSxq5bqCP9k9lPNM7lfMF?=
=?us-ascii?Q?GvS/bDKXvgD2C9g5qvh/6MgQBmbXijP5ox5xePZTBoCnWYxZKhg/yBZ07vtp?=
=?us-ascii?Q?5irGFQQhRb69DdGe3crb4XI5CRq8gIcm4aeAUViE3Uxe1aZivAZVS7b1NXZI?=
=?us-ascii?Q?RltEq1HXq6vuuUfLfRijainR5bRXTgRmfmHYme6Xtl4qvmNOb2CHQ7hD3b8J?=
=?us-ascii?Q?VA5k/aSrsqyoXbJRrGsy3W7BM6lvRk7jYPo+zjkSTgQtxaIFvxmXZlQMThqn?=
=?us-ascii?Q?Plt0XK/8rE+eZbal5+j1xQd6LLA1eaKcwuKorGZQXSPtOTgSIKg/c/a9Ma9f?=
=?us-ascii?Q?TTog5r3+7Yj+MKQgC3CICziD/39VReTWpbrlksjchrG0mgniN3z/6IYuCRY2?=
=?us-ascii?Q?5UQKkEYYMoxfRl98qiCBSHc3nogv?=
X-Forefront-Antispam-Report:
CIP:172.179.146.41;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:104.47.53.36;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(61400799027);DIR:OUT;SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
NY8NAfnSjnGmunwEE1ByyaAv8ChGS2wdJ5egQFwMw9HoQzI4aNX2u+X+MclJj4vFU3Bi+JrH2EkBcg+lWT+vg69gOPjdUPAe91TOLB4gENtuSUyTjijBbiKIqe9USw1vKrob1kmH3cwpusMsqQ8jZWVir9dVi5zvUFAGaGFSLlvvtdkBk8qc/h3IwJ1JaTWAbvymLIDfPyCM9ut/0t1UyCcHBDt/NptX7lW+llvP7CUOCOzudx7F4v6czUW1KYl2fEzAnFeUfYRo0gGJgEVKCD6GAS28nzRo8H8EyBK7zvZt5pyy5QHak1Wz8mhJIU0DJ45C/EVgi+SvXl1T9ZagbC2InOdA2nmCqNPNbz272aizHkd0YmiY+S4bhgnsa7aSad7KaKoD/hRI9c63MQR+IRhKV1Yukj0xC+aCM1g0CVPdtOYfQUwujNaVx8kANKCmKjLcsyCEhaA/5MUotYZepPqvW7t3zRTFE2W41O770oGqczpNjk0HtpAfBLBXeYBY
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Sep 2024 20:42:31.7070
(UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ae339a39-51d3-4824-723f-08dcd36b6cb3
X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47;Ip=[172.179.146.41];Helo=[104.47.53.36]
X-MS-Exchange-CrossTenant-FromEntityHeader: Internet
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource:
TreatMessagesAsInternal-IA2PR21MB4323.namprd21.prod.outlook.com
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA2PR21MB4323

--a737a3b9-5ea5-40e8-a971-a67ecbb2993e
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

This is an aggregate TLS report from microsoft.com

Microsoft respects your privacy. Review our online Privacy Statement<= /em>

Microsoft Corporation
One Microsoft Way
Redmond, WA, USA 98052

--a737a3b9-5ea5-40e8-a971-a67ecbb2993e
Content-Type: application/tlsrpt+gzip
Content-Description:
microsoft.com!fake.domain.name!1726012800!1726099199!111111111111111111.json.gz
Content-Disposition: attachment;
filename="microsoft.com!fake.domain.name!1726012800!1726099199!111111111111111111.json.gz"
Content-Transfer-Encoding: base64

H4sIAAAAAAAEAH2STWvDMAyG/0rwdXVwPpa1Pg123qk9dZThOU4wxFawldKs5L9PSdsxBi0ILPt9
ZL8SPjMIrfL2W6EFz71yhkn2bnWACA0mbxB6CIvIVqxWaHhQviXozCKqgHw+Q7uU5SIvudjwLNsJ
IZfYU5Xx9R0qL+TzhmLPphXT4FFp5NY3QBh2MfTIPQTTd+Oru1lKNTi6lE6BXrc1oVlRvIiqzMts
vc43oqqKpzi6LwtoddpYonvorLYmMvlxvmzGuYNLxnHsZ2MR4w0decRgfUs8O5oQqX2ZbHfbY0aE
g9rIxJDNoM28P8lExd6d0i5tAdrOXD0uQodZ+kjNH6rFQ7W8o6rTp2rJYyXKtRDs8NtVDU5ZP/f6
dz40/Dg4p8IyFARUHY+D1ibGZqCU1vlzaBg8MpmvrkijbDcE818X03SYfgCxXYk2WAIAAA==

--a737a3b9-5ea5-40e8-a971-a67ecbb2993e--

[makuartur]

@seanthegeek, is the answer provided appropriate?

[seanthegeek]

Yes. The problem is a mistake in the email headers when Microsoft is generating the tlsrpt email. You can verify this by pasting the content you provided into a .eml file and then opening in in an email reader like Thunderbird. The email content will be blank.

The mistake is here, where the Content-Type header is split over two lines without indenting the second line.

Content-Type: multipart/report;
boundary="a737a3b9-5ea5-40e8-a971-a67ecbb2993e"; report-type=tlsrpt

If you combine the content into one line

Content-Type: multipart/report; boundary="a737a3b9-5ea5-40e8-a971-a67ecbb2993e"; report-type=tlsrpt

Or indent the second line

Content-Type: multipart/report;
  boundary="a737a3b9-5ea5-40e8-a971-a67ecbb2993e"; report-type=tlsrpt

Then the email will be successfully parsed by Thunderbird and parsedmarc.

I'll see if I can find someone at Microsoft to address this.

[seanthegeek]

@makuartur Actually, looking at this again, it looks like GitHub removed all of the indents when you pasted in the sample as test, Please save the sample as a file, then drag and prop that file into the comment box to make it an attachment.

[makuartur]

mail.txt
Of course, here it is in the file.
@seanthegeek

[seanthegeek]

Hmm. the parsedmarc CLI parsed the email correctly, so I'm not sure why it would be moved to the invalid folder. Can you try updating to the latest release of parsedmarc, moving one of the emails from the invalid folder back to the inbox and see if the same thing happens again?