===== IDP v4 as proxy =====

It happens that SP vendors don't provide Discocery Service/WAYF SP initiated SSO  (has we are used in academic/research ecosystem)
so they ask us to register as many IDP as we have universities/school in our group of internals federated IDPs .

to circonvent that constraint, we decided to expose and register only one IDP (one to one) with those vendors. this IDP will act as a proxy between the vendor SP and internals IDPs . that IDP "in the middle" will act as a IDP for the SP and as a SP regarding internals IDPs . 

reference : 
  * https://wiki.shibboleth.net/confluence/display/KB/Using+SAML+Proxying+to+another+IdP
  * https://wiki.shibboleth.net/confluence/display/IDP4/SAMLAuthnConfiguration

Matching above doc terminology,  here is how we interpreted it: 
  - "Original IdP EntityID: https://idp.example.ac.uk/entity" beeing the IDP-proxy that will act as a SP to schools IDPs, so declare its metadata the <SPSSODescriptor .. element to act as a SP in the  internal federation
  - "Upstream IdP EntityID: https://upstream.idp/entity" beeing the actual school IDP where nothing is changed, all configuration regarding : SAML flow, attribute-filter, subject canonicalisation beeing done in 1

===== metadata consideration =====

The idea is to  present one set of metadata to our internal federation/IdP's and one set of metadata to the vendor SP, which both happened to use the same entityID (our IDPproxy) .  We would not be publishing both versions in the internal federation metadata.  The federation never needs to know, technically, that we're a proxy.  Our  IDPproxy  just look like another service.

==== SPSSODescriptor ====

act as a SP in internal Federation, metadata looks like : 

<code>
<md:EntityDescriptor ID="_5b39720f724ef5a082af7739d48bf4da3c5523d5" entityID="https://idproxy.domain.fr/idp/shibboleth" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata">


 <md:SPSSODescriptor protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
    <md:KeyDescriptor use="signing">
    ...    
  <ds:X509Certificate>
    ...
 <md:AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://idproxy.domain.fr/idp/profile/Authn/SAML2/POST/SSO" index="0"/>

  </md:SPSSODescriptor>

</code>


==== IDPSSODescriptor ====

regarding our vendor SP , we present the traditional installed based IDP metadata with the IDPs SSO bindings endpoints (metadata/idp-metadata.xml)  

<code>
<EntityDescriptor  xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:shibmd="urn:mace:shibboleth:metadata:1.0" xmlns:xml="http://www.w3.org/XML/1998/namespace" xmlns:mdui="urn:oasis:names:tc:SAML:metadata:ui" xmlns:req-attr="urn:oasis:names:tc:SAML:protocol:ext:req-attr" entityID="https://idproxy.domain.fr/idp/shibboleth">

    <IDPSSODescriptor protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol urn:oasis:names:tc:SAML:1.1:protocol urn:mace:shibboleth:1.0">
....
 <SingleSignOnService Binding="urn:mace:shibboleth:1.0:profiles:AuthnRequest" Location="https://idproxy.domain.fr/idp/profile/Shibboleth/SSO"/>
        <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" req-attr:supportsRequestedAttributes="true" Location="https://idproxy.domain.fr/idp/profile/SAML2/Redirect/SSO"/>
        <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" req-attr:supportsRequestedAttributes="true" Location="https://idproxy.domain.fr/idp/profile/SAML2/POST/SSO"/>
        <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST-SimpleSign" req-attr:supportsRequestedAttributes="true" Location="https://idproxy.domain.fr/idp/profile/SAML2/POST-SimpleSign/SSO"/>
   </IDPSSODescriptor>
...
    <AttributeService Binding="urn:oasis:names:tc:SAML:1.0:bindings:SOAP-binding" Location="https://idproxy.domain.fr:8443/idp/profile/SAML1/SOAP/AttributeQuery"/>

    </AttributeAuthorityDescriptor>

</EntityDescriptor>
       
</code>


==== MetadataFilter ====

Beware , in a traditional IDP role, the load of metadata providers (in conf/metadata-providers.xml) usually comes with a "EntityRoleWhiteList" that loads only SPs from the Federation Metadatas, but here we act as a SP so we really need to load IDPs roles ! 

<code>
<!-- comment Role to load only SP in this context 
 <MetadataFilter xsi:type="EntityRoleWhiteList">
            <RetainedRole>md:SPSSODescriptor</RetainedRole>
        </MetadataFilter> -->
</code>

===== use SAML flow =====

tell the IDP (in conf/idp.properties)  to use the SAML flow 

<code>
#idp.authn.flows=Password
idp.authn.flows=SAML
</code>

ref: https://wiki.shibboleth.net/confluence/display/IDP4/SAMLAuthnConfiguration
===== IDPproxy Discovery Service / WAYF =====

ref : https://wiki.shibboleth.net/confluence/display/IDP4/AuthenticationConfiguration#AuthenticationConfiguration-AdvancedTopics 

permit Discovery Service to redirect to our Federation "switchWayf" DS service (Wayf.php) 


1) comment my previous test that redirected SSO to one specific IDP in authn/saml-authn-config.xml
<code> <!-- <bean id="shibboleth.authn.SAML.discoveryFunction" parent="shibboleth.Functions.Constant"
        c:target="https://upstreamIDP1/idp/shibboleth" /> --></code>
        
2) point authn.discoveryURL in authn/discovery-config.xml to the DS/WAYF URL 
<code> <bean id="shibboleth.authn.discoveryURL" class="java.lang.String"
        c:_0="https://shibds.domain.fr/wayfds/WAYF.php" /></code>
        
===== Subject Principal =====

from : https://wiki.shibboleth.net/confluence/display/IDP4/SAMLAuthnConfiguration

to resume Subject Canonicalization sub chapter of that doc:

// "there is a requirement to process the AuthenticationResult into a normalized principal name,...this isn't a trivial thing to do. there is no simple "username" to rely on and the system requires some dedicated configuration to know what to do."//

So I followed the doc by defining in attribute-resolver.xml the mail attribute as beeing of type="SubjectDerivedAttribute":

<code>
<AttributeDefinition id="canonicalName" xsi:type="SubjectDerivedAttribute"
    forCanonicalization="true" principalAttributeName="mail" />
</code>
then select mail from my upstreamIDP issuer in attribute-filter.xml 

<code>
<PolicyRequirementRule xsi:type="Issuer" value="https://upstreamIDP.domain.fr/idp/shibboleth" />
                <AttributeRule attributeID="mail" permitAny="true" />
</code>

then , enabling post-Login c14n flow in subject-c14n.xml : <bean id="c14n/attribute" parent="shibboleth.PostLoginSubjectCanonicalizationFlow" />

Finaly associate the id=canonicalName for "the list of attributes to resolve for normalizing the subject" in attribute-sourced-subject-c14n-config.xml

<code>
<util:list id="shibboleth.c14n.attribute.AttributesToResolve">
        <value>canonicalName</value>
    </util:list>

<util:list id="shibboleth.c14n.attribute.AttributeSourceIds">
        <value>canonicalName</value>
    </util:list>
</code>

Now the canonicalization process does include the c14n/attribute flow, see logs below with 3 canonicalization flows (there were the 2 defaults before; c14n/x500 and c14n/simple) 

<code>
2021-02-03 22:41:08,282 -- DEBUG [net.shibboleth.idp.authn.impl.PopulateSubjectCanonicalizationContext:75] - Profile Action PopulateSubjectCanonicalizationContext: Installing 3 canonicalization flows into SubjectCanonicalizationContext
</code>

Then I can see that the mail attribute seems well passThrough upstreamIDP to IDPproxy by my "canonicalName" attribute definition:

<code>
2021-02-03 22:41:08,397 -- DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.ContextDerivedAttributeDefinition:179] - SubjectDerivedAttributeDefinition canonicalName Generated 1 values
2021-02-03 22:41:08,398 -- DEBUG [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:139] - Attribute Definition 'canonicalName': produced an attribute with the following values [StringAttributeValue{value=jehan.procaccia@imtbs-tsp.eu}]
2021-02-03 22:41:08,400 -- DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:519] - Attribute Resolver 'ShibbolethAttributeResolver': Attribute 'canonicalName' has 1 values after post-processing
</code>

but then, next log line,  it fails : 

<code>
2021-02-03 22:41:08,408 -- ERROR [net.shibboleth.idp.authn:-2] - Uncaught runtime exception
java.lang.NullPointerException: null
	at net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl.collectExportingDataConnectors(AttributeResolverImpl.java:542)
2021-02-03 22:41:08,413 -- INFO [net.shibboleth.idp.authn.impl.SelectAuthenticationFlow:142] - Profile Action SelectAuthenticationFlow: Moving incomplete flow authn/SAML to intermediate set
</code>

That was a bug in 4.0.1, shib-user list responded : // you can't use the exporting feature with connectors that produce no data until it's fixed. Just don't do the exporting trick for now if they occasionally produce no results.//

Indeed , I don't use ldap attributes in the context of the IDPProxy, it just pass-through attributes sent from upstreamIDPs, 
so remove "myLDAP" Data connector from attribute-resolver.xml as it not necessary in that context (which produced no value, and hence the bug)

then back to idp-preocess.log, the flow continues : 

<code>
2021-02-04 09:13:52,444 -- DEBUG [net.shibboleth.idp.authn.impl.AttributeSourcedSubjectCanonicalization:171] - Profile Action AttributeSourcedSubjectCanonicalization: Using attribute canonicalName string value jehan.procaccia@imtbs-tsp.eu as input to transforms
2021-02-04 09:13:52,445 -- DEBUG [net.shibboleth.idp.authn.AbstractSubjectCanonicalizationAction:218] - Profile Action AttributeSourcedSubjectCanonicalization: trimming whitespace of input string 'jehan.procaccia@imtbs-tsp.eu'
2021-02-04 09:13:52,451 -- DEBUG [net.shibboleth.idp.authn.impl.FinalizeAuthentication:118] - Profile Action FinalizeAuthentication: Canonical principal name was established as 'jehan.procaccia@imtbs-tsp.eu'
2021-02-04 09:13:52,456 -- DEBUG [net.shibboleth.idp.session.impl.UpdateSessionWithAuthenticationResult:222] - Profile Action UpdateSessionWithAuthenticationResult: Creating new session for principal jehan.procaccia@imtbs-tsp.eu
</code>

===== pass-though attributes =====

ThenI encontered an issue regarding pass-through attributes, only the subject Princicpal was transmetted to the SP :

<code>
2021-02-04 09:13:52,451 -- DEBUG [net.shibboleth.idp.authn.impl.FinalizeAuthentication:118] - Profile Action FinalizeAuthentication: Canonical principal name was established as 'jehan.procaccia@imtbs-tsp.eu'
2021-02-04 09:13:52,500 - 157.159.52.132 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:139] - Attribute Definition 'uid': produced an attribute with the following values [StringAttributeValue{value=jehan.procaccia@imtbs-tsp.eu}]
</code>

But the other ones resolved before => eduPersonPrincipalName, eduPersonScopedAffiliation, uid ; 

<code>
2021-02-04 09:13:52,301 -- DEBUG [net.shibboleth.idp.attribute.filter.AttributeFilterPolicy:153] - Attribute Filter Policy 'saml-proxy-pass-through'  Applying attribute filter policy to current set of attributes: [mail, eduPersonPrincipalName, eduPersonScopedAffiliation, uid]
</code>

got lost in the process . 

The solution was in the Doc :  https://wiki.shibboleth.net/confluence/display/IDP4/SubjectDataConnector

A new  dataConnector for the IPD-proxy (attribute-resolver.xml) was needed :

<code>
<DataConnector id="passthroughAttributes" xsi:type="Subject" exportAttributes="mail givenName sn eduPersonAffiliation eduPersonPrincipalName" />
</code>

there is no need to add to that Dataconnector a derivated AttributeDefinition , which I've done in the 1st place but then commented and it still worked fine

:!: not needed :!:
<code>
<!-- <AttributeDefinition xsi:type="Simple" id="sn">
    <InputDataConnector ref="passthroughAttributes" attributeNames="sn" />
</AttributeDefinition> --> 
</code>


===== Raw shibboleth-users ML threads =====

All this was possible thanks to the Mailing list shibboleth-users , cf this thread for details 
  * https://marc.info/?t=161193758600001&r=1&w=2