Redshift:需要超长查询的效率帮助

时间:2017-08-28 20:52:48

标签: amazon-redshift

首先关闭:这是用aginity编写的redshift查询

对于我正在制作的模型,我需要一些超长查询的帮助。在我上一节第10节之前,一切都很有效,我必须在那里做一些工会。我可以运行,但不能引用临时表 Inquiry7 Inquiry8 。如果我运行整个查询并在最后放置Select * from Inquiry6,它就会运行。但是,如果我运行整个查询并在最后添加Select * from Inquiry7,我会开始得到奇怪的错误,例如连接已断开。

我还应该注意,我没有redshift的写访问权限。

有什么特别突出的我可以做的,以提高效率吗?由于篇幅限制,我只发布了代码的后半部分。

--------------------------------------------------------------------------------------------------------------------------------------
--Part 6: Appending SQL, SQL Reviewed, and SQO dates
--------------------------------------------------------------------------------------------------------------------------------------

--6.1 SQL Date
sql_date as (
                Select a.*, convert_timezone('UTC', 'US/Pacific',b.CreatedDate) as SQL_Date
                From part5_final a
                Left Outer Join salesforce_opportunity b on a.convertedopportunityid = b.id
            ),

--6.2 SQL Reviewed Date and Value
sql_reviewed as (
                    Select a.*,
                           First_Value (b.CreatedDate) Over (Partition By a.ConvertedContactId, a.MQL_Date ORDER BY b.CreatedDate ASC rows between unbounded preceding and unbounded following) as SQL_reviewed_date,
                           First_Value (b.NewValue) Over (Partition By a.ConvertedContactId, a.MQL_Date ORDER BY b.CreatedDate ASC rows between unbounded preceding and unbounded following) as SQL_reviewed_value
                    From sql_date a Left Outer Join (Select a.OpportunityId, convert_timezone('UTC', 'US/Pacific',a.CreatedDate) as CreatedDate, a.NewValue From salesforce_opportunityfieldhistory a Where a.field = 'StageName' and a.OldValue = 'Sales Qualified Lead') b on a.ConvertedOpportunityId = b.OpportunityId
                ),

--6.3 Discovery Date
Discovery_Date as (
                    Select a.*,
                           First_Value (b.CreatedDate) Over (Partition By a.ConvertedContactId, a.MQL_Date ORDER BY b.CreatedDate ASC rows between unbounded preceding and unbounded following) as Discovery_date
                    From sql_reviewed a Left Outer Join (Select a.OpportunityId, convert_timezone('UTC', 'US/Pacific',a.CreatedDate) as CreatedDate, a.NewValue From salesforce_opportunityfieldhistory a Where a.field = 'StageName' and a.NewValue not in ('Sales Qualified Lead', 'Closed - SQL Rejected')) b on a.ConvertedOpportunityId = b.OpportunityId
                  ),     

--6.4 SQO Date
SQO_Date as (
                Select a.*,
                       First_Value (b.CreatedDate) Over (Partition By a.ConvertedContactId, a.MQL_Date ORDER BY b.CreatedDate ASC rows between unbounded preceding and unbounded following) as SQO_date
                From Discovery_Date a Left Outer Join (Select a.OpportunityId, convert_timezone('UTC', 'US/Pacific',a.CreatedDate) as CreatedDate, a.NewValue From salesforce_opportunityfieldhistory a Where a.field = 'StageName' and a.NewValue not in ('Sales Qualified Lead', 'Discovery', 'Closed - SQL Rejected', 'Closed - SQL Unqualified')) b on a.ConvertedOpportunityId = b.OpportunityId
             ),

--6.5 Reorg and dedupe           
Part6_Final as (
                    Select leadid, convertedcontactid, mql_date, mql_reviewed_date, mql_reviewed_value, sal_date, lead_closed_date, lead_closed_value, converteddate, convertedopportunityid, sql_date, sql_reviewed_date, sql_reviewed_value, discovery_date, sqo_date, opportunitycloseddate, opportunitystage
                    From SQO_Date
                    Group By leadid, convertedcontactid, mql_date, mql_reviewed_date, mql_reviewed_value, sal_date, lead_closed_date, lead_closed_value, converteddate, convertedopportunityid, sql_date, sql_reviewed_date, sql_reviewed_value, discovery_date, sqo_date, opportunitycloseddate, opportunitystage
                ),
--102 Seconds

--------------------------------------------------------------------------------------------------------------------------------------
--Part 7: Appending Offer: we are going to build a table of MQL_Status__c changes, then do a date diff to calculate the time from MQL
--        Date, taking the min result as our offer. 
--------------------------------------------------------------------------------------------------------------------------------------              

--7.1 This grabs the values for the first MQL_Status__c change on the lead history table
offer1 as (
            Select a.leadid, 
                   First_Value (convert_timezone('UTC', 'US/Pacific',a.CreatedDate)) Over (Partition By a.LeadId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as CreatedDate,
                   First_Value (a.OldValue) Over (Partition By a.LeadId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as OldValue,
                   First_Value (a.NewValue) Over (Partition By a.LeadId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as NewValue
            From salesforce_leadhistory a
            Where a.Field = 'MQL_Status__c'
           ),

--7.2 There are instances where a lead was created with MQL_Status__c populated. In order to accurately identify the first
--    change, we will take the Old Value and the lead created date for those 

offer2 as (
            Select a.leadid, CASE WHEN a.oldvalue <> '' THEN convert_timezone('UTC', 'US/Pacific',b.CreatedDate) ELSE a.CreatedDate END as CreatedDate,
                             CASE WHEN a.oldvalue <> '' THEN a.oldValue ELSE a.NewValue END as NewValue
            From offer1 a
            Inner Join salesforce_lead b on a.leadid = b.id
           ),

--7.3 This grabs the values for the first MQL_Status__c change on the contact history table
offer3 as (
            Select a.contactid, 
                   First_Value (convert_timezone('UTC', 'US/Pacific',a.CreatedDate)) Over (Partition By a.ContactId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as CreatedDate,
                   First_Value (a.OldValue) Over (Partition By a.ContactId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as OldValue,
                   First_Value (a.NewValue) Over (Partition By a.ContactId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as NewValue
            From salesforce_contacthistory a
            Where a.Field = 'MQL_Status__c'
           ),

--7.4 There are instances where a lead was created with MQL_Status__c populated. In order to accurately identify the first
--    change, we will take the Old Value and the lead created date for those 
Offer4 as (
            Select a.contactid, CASE WHEN a.oldvalue <> '' THEN convert_timezone('UTC', 'US/Pacific',b.CreatedDate) ELSE a.CreatedDate END as CreatedDate,
                                CASE WHEN a.oldvalue <> '' THEN a.oldValue ELSE a.NewValue END as NewValue
            From offer3 a
            Inner Join salesforce_contact b on a.contactid = b.id
           ),

--7.5 combining results from 6.2 and 6.4. Additionally filling in all other contact and lead offer changes. Also, dedupe. 
offer5 as (
            Select a.PersonId, a.CreatedDate, a.NewValue
            From (
                    Select CASE WHEN b.ConvertedContactId <> '' THEN b.ConvertedContactId else a.LeadId END as PersonId, a.CreatedDate, a.NewValue
                    From offer2 a
                    Inner Join salesforce_lead b on a.leadid = b.id

                    UNION ALL

                    Select a.ContactId as PersonId, a.CreatedDate, a.NewValue
                    From offer4 a

                    UNION ALL

                    Select a.leadid as PersonId, (convert_timezone('UTC', 'US/Pacific',a.CreatedDate)) as createddate, a.newvalue
                    From salesforce_leadhistory a
                    Where a.Field = 'MQL_Status__c'

                    UNION ALL

                    Select a.contactid as PersonId, (convert_timezone('UTC', 'US/Pacific',a.CreatedDate)) as createddate, a.newvalue
                    From salesforce_contacthistory a
                    Where a.Field = 'MQL_Status__c'
                 ) a
            Group By a.PersonId, a.CreatedDate, a.NewValue
          ),

--7.6 Identification of correct offer per MQL 
offer6 as (
            Select a.*, 
                   First_Value (b.NewValue) Over (Partition By ISNULL(a.ConvertedContactId, a.LeadId), a.MQL_Date ORDER BY abs(datediff(min,a.mql_date, b.createddate)) ASC rows between unbounded preceding and unbounded following) as Offer
            From Part6_Final a 
            Left Outer Join offer5 b on ISNULL(a.ConvertedContactId, a.LeadId) = b.PersonId
            --Group By a.LeadId, a.ConvertedContactId, a.MQL_Date, a.MQL_Reviewed_Date, a.MQL_Reviewed_Value, a.SAL_Date, a.Lead_Closed_Date, a.Lead_Closed_Value, a.ConvertedDate, a.ConvertedOpportunityId, a.SQL_Date, a.SQL_Reviewed_Date, a.SQL_Reviewed_Value, a.Discovery_Date, a.SQO_Date, a.OpportunityClosedDate, a.OpportunityStage
          ),
--7.7 Dedupe
offer7 as (
            Select *
            From offer6 a
            Group By a.LeadId, a.ConvertedContactId, a.MQL_Date, a.MQL_Reviewed_Date, a.MQL_Reviewed_Value, a.SAL_Date, a.Lead_Closed_Date, a.Lead_Closed_Value, a.ConvertedDate, a.ConvertedOpportunityId, a.SQL_Date, a.SQL_Reviewed_Date, a.SQL_Reviewed_Value, a.Discovery_Date, a.SQO_Date, a.OpportunityClosedDate, a.OpportunityStage, a.Offer
          ),

--7.8 Cases where a change does not appear in the history table, but the field is populated on the record, this query fills those gaps in. 
Part7_Final as (
            Select a.LeadId, a.ConvertedContactId, a.MQL_Date, a.MQL_Reviewed_Date, a.MQL_Reviewed_Value, a.SAL_Date, a.Lead_Closed_Date, a.Lead_Closed_Value, a.ConvertedDate, a.ConvertedOpportunityId, a.SQL_Date, a.SQL_Reviewed_Date, a.SQL_Reviewed_Value, a.Discovery_Date, a.SQO_Date, a.OpportunityClosedDate, a.OpportunityStage, 
                   CASE WHEN a.Offer IS NULL AND a.ConvertedContactId is NULL THEN c.MQL_Status__c
                        WHEN a.Offer IS NULL and a.ConvertedContactId is NOT NULL THEN b.MQL_Status__c
                        ELSE a.Offer
                   END as Offer
            From Offer7 a
            Left Outer Join salesforce_contact b on a.convertedcontactid = b.id
            Left Outer Join salesforce_lead c on a.leadid = c.id
           ),
--160 seconds
--21,506 rows

--------------------------------------------------------------------------------------------------------------------------------------
--Part 8: Lead Source / Marketing Channel
--------------------------------------------------------------------------------------------------------------------------------------      
--8.1 Determine previous MQL date so we have a start end end date for determining correct last touch attribution.
mktgchannel1 as (
                    Select a.*, 
                           ISNULL(Last_Value (b.mql_date) Over (Partition By ISNULL(a.ConvertedContactId, a.LeadId), a.MQL_Date ORDER BY b.mql_date ASC rows between unbounded preceding and unbounded following), '10/1/2016') as PrevMQLDate
                    From Part7_Final a
                    Left Outer Join ( 
                                        Select a.leadid, a.convertedcontactid, a.mql_date
                                        From Part7_Final a 
                                        Group By a.leadid, a.convertedcontactid, a.mql_date
                                    ) b on ISNULL(a.ConvertedContactId, a.LeadId) = ISNULL(b.ConvertedContactId, b.LeadId) AND a.mql_date > b.mql_date
               ),
--209 seconds

--8.2 Building marketing channel source table from Bizible data
mktgchannel2 as (
                    Select CASE WHEN b.bizible2__Contact__c <> '' THEN b.bizible2__Contact__c ELSE b.bizible2__Lead__c END as PersonId, convert_timezone('UTC', 'US/Pacific',a.bizible2__Touchpoint_Date__c) as TouchpointDate, a.Id
                    From salesforce_bizible2__bizible_touchpoint__c a
                    Inner Join salesforce_bizible2__bizible_person__c b on a.bizible2__Bizible_Person__c = b.Id
                    Where a.bizible2__Touchpoint_Position__c like '%Form%'
                ),
--8.3 Dedupe 
mktgchannel3 as (
                    Select *
                    From (
                            Select a.*, 
                                   Last_Value (b.id) Over (Partition By ISNULL(a.ConvertedContactId, a.LeadId), a.MQL_Date ORDER BY b.touchpointdate ASC rows between unbounded preceding and unbounded following) as TouchpointId,
                                   Last_Value (b.TouchpointDate) Over (Partition By ISNULL(a.ConvertedContactId, a.LeadId), a.MQL_Date ORDER BY b.touchpointdate ASC rows between unbounded preceding and unbounded following) as TouchpointDate
                            From mktgchannel1 a
                            Left Outer Join mktgchannel2 b on ISNULL(a.ConvertedContactId, a.LeadId) = b.PersonId and b.TouchpointDate between a.PrevMQLDate and a.MQL_Date
                          )a
                    Group By a.leadid, a.convertedcontactid, a.mql_date, a.mql_reviewed_date, a.mql_reviewed_value, a.sal_date, a.lead_closed_date, a.lead_closed_value, a.converteddate, a.convertedopportunityid, a.sql_date, a.sql_reviewed_date, a.sql_reviewed_value, a.discovery_date, a.sqo_date, a.opportunitycloseddate, a.opportunitystage, a.offer, a.prevmqldate, a.touchpointid, a.touchpointdate
                 ),
--226 seconds        

--8.4 Building attribution marketing channel source table from Bizible data
mktgchannel4 as (
                    Select a.bizible2__opportunity__c as opportunityid, id as attributionid, convert_timezone('UTC', 'US/Pacific',a.bizible2__Touchpoint_Date__c) as attributiondate
                    From salesforce_bizible2__bizible_attribution_touchpoint__c a
                    Where bizible2__form_url__c <> ''
                ),

--8.5 Appending attribution id / date
mktgchannel5 as (
                    Select a.*,
                           Last_Value (b.attributionid) Over (Partition By ISNULL(a.ConvertedContactId, a.LeadId), a.MQL_Date ORDER BY b.attributiondate ASC rows between unbounded preceding and unbounded following) as AttributionId,
                           Last_Value (b.attributionDate) Over (Partition By ISNULL(a.ConvertedContactId, a.LeadId), a.MQL_Date ORDER BY b.attributiondate ASC rows between unbounded preceding and unbounded following) as AttributionDate
                    From mktgchannel3 a
                    Left Outer Join mktgchannel4 b on a.convertedopportunityid = b.opportunityid AND attributiondate between a.prevmqldate and mql_date
                ),
--8.6 Dedupe
mktgchannel6 as (
                    Select *
                    From mktgchannel5 a
                    Group by a.leadid, a.convertedcontactid, a.mql_date, a.mql_reviewed_date, a.mql_reviewed_value, a.sal_date, a.lead_closed_date, a.lead_closed_value, a.converteddate, a.convertedopportunityid, a.sql_date, a.sql_reviewed_date, a.sql_reviewed_value, a.discovery_date, a.sqo_date, a.opportunitycloseddate, a.opportunitystage, a.offer, a.prevmqldate, a.touchpointid, a.touchpointdate, a.attributionid, a.attributiondate
                ),

--8.7 Determining whether the touchpoint or attribution touchpoint is more relevent to the last touch model
mktgchannel7 as (
                    Select a.leadid, a.convertedcontactid, a.mql_date, a.mql_reviewed_date, a.mql_reviewed_value, a.sal_date, a.lead_closed_date, a.lead_closed_value, a.converteddate, a.convertedopportunityid, a.sql_date, a.sql_reviewed_date, a.sql_reviewed_value, a.discovery_date, a.sqo_date, a.opportunitycloseddate, a.opportunitystage, a.offer, a.prevmqldate,
                           CASE WHEN a.touchpointid IS NULL and a.attributionid IS NOT NULL THEN a.attributionid
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NULL THEN a.touchpointid
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NOT NULL AND a.touchpointdate > a.attributiondate THEN a.touchpointid
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NOT NULL and a.touchpointdate < a.attributiondate THEN a.attributionid
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NOT NULL and a.touchpointdate = a.attributiondate THEN a.touchpointid
                                WHEN a.touchpointid IS NULL and a.attributionid IS NULL THEN NULL
                           END as TouchpointId,
                            CASE WHEN a.touchpointid IS NULL and a.attributionid IS NOT NULL THEN a.attributiondate
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NULL THEN a.touchpointdate
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NOT NULL AND a.touchpointdate > a.attributiondate THEN a.touchpointdate
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NOT NULL and a.touchpointdate < a.attributiondate THEN a.attributiondate
                                WHEN a.touchpointid IS NOT NULL and a.attributionid IS NOT NULL and a.touchpointdate = a.attributiondate THEN a.touchpointdate
                                WHEN a.touchpointid IS NULL and a.attributionid IS NULL THEN NULL
                           END as TouchpointDate
                    From mktgchannel6 a
                ),

--8.8 Bringing in the marketing channel and form url from bizible data
Part8_Final as (
                    Select a.leadid, a.convertedcontactid as contactid, ISNULL(a.touchpointdate, a.mql_date) as inquiry_date, a.mql_date, a.mql_reviewed_date, a.mql_reviewed_value, a.sal_date, a.lead_closed_date, a.lead_closed_value, a.converteddate as converted_date, a.convertedopportunityid as opportunityid, a.sql_date, a.sql_reviewed_date, a.sql_reviewed_value, a.discovery_date, a.sqo_date, a.opportunitycloseddate as opportunity_closed_date, a.opportunitystage as opportunity_stage, a.offer, a.prevmqldate as prev_mql_date, a.touchpointid, a.touchpointdate as touchpoint_date, 
                    ISNULL(b.bizible2__marketing_channel__c, c.bizible2__marketing_channel__c) as Lead_Source, 
                    ISNULL(b.bizible2__form_url__c, c.bizible2__form_url__c) as Lead_Source_URL
                    From mktgchannel7 a
                    Left Outer Join salesforce_bizible2__bizible_touchpoint__c b on a.touchpointid = b.id
                    Left Outer Join salesforce_bizible2__bizible_attribution_touchpoint__c c on a.touchpointid = c.id
                ),
--282 secs              

--------------------------------------------------------------------------------------------------------------------------------------
--Part 9: New leads that haven't had a status change yet
--------------------------------------------------------------------------------------------------------------------------------------  
--9.1 identification of MQLs that are waiting to be evaluated and do not appear in the model thus far
newlead1 as (
                Select a.Id as LeadId, convert_timezone('UTC', 'US/Pacific',a.CreatedDate) as LeadCreatedDate
                From salesforce_lead a
                Left Outer Join (
                                  Select Distinct LeadId
                                  From Part8_Final a
                                ) b on a.Id = b.LeadId
                Where a.Status = 'New'
                AND a.MQL_Status__c <> ''
                AND a.MQL_Status__c IS NOT NULL
                AND b.LeadId IS NULL
             ),


--9.2 This takes the leads identified in 9.1 and looks for the first offer change
newlead2 as (
                Select b.LeadId, b.LeadCreatedDate,
                       First_Value (convert_timezone('UTC', 'US/Pacific',a.CreatedDate)) Over (Partition By b.LeadId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as CreatedDate,
                       First_Value (a.OldValue) Over (Partition By b.LeadId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as OldValue,
                       First_Value (a.NewValue) Over (Partition By b.LeadId ORDER BY convert_timezone('UTC', 'US/Pacific',a.CreatedDate) ASC rows between unbounded preceding and unbounded following) as NewValue
                From (Select * From salesforce_leadhistory Where field = 'MQL_Status__c') a 
                Right Outer Join newlead1 b on a.leadid = b.leadid 

            ),

--9.3 dedupe
newlead3 as (
              Select *
              From newlead2 
              Group By leadid, leadcreateddate, createddate, oldvalue, newvalue
            ),

--9.4 Identifying the appropriate offer and offer date to use
newlead4 as (
                Select a.leadid, 
                       CASE WHEN a.oldvalue = '' THEN a.createddate
                            WHEN a.oldvalue <> '' THEN a.leadcreateddate
                            WHEN a.oldvalue IS NULL THEN a.leadcreateddate
                       END as MQL_Date,
                       CASE WHEN a.oldvalue = '' THEN a.newvalue
                            WHEN a.oldvalue <> '' THEN a.oldvalue
                            WHEN a.oldvalue IS NULL THEN b.MQL_Status__c 
                        END as Offer
                From newlead3 a
                Inner Join salesforce_lead b on a.leadid = b.id
            ),

--9.5 Indentifying the bizible touchpoint to use
newlead5 as (
                Select a.*,
                       Last_Value (c.id) Over (Partition By a.LeadId, a.MQL_Date ORDER BY convert_timezone('UTC', 'US/Pacific',c.bizible2__touchpoint_date__c) ASC rows between unbounded preceding and unbounded following) as TouchpointId,
                       Last_Value (convert_timezone('UTC', 'US/Pacific',c.bizible2__touchpoint_date__c)) Over (Partition By a.LeadId, a.MQL_Date ORDER BY convert_timezone('UTC', 'US/Pacific',c.bizible2__touchpoint_date__c) ASC rows between unbounded preceding and unbounded following) as Touchpoint_Date,
                       Last_Value (bizible2__marketing_channel__c) Over (Partition By a.LeadId, a.MQL_Date ORDER BY convert_timezone('UTC', 'US/Pacific',c.bizible2__touchpoint_date__c) ASC rows between unbounded preceding and unbounded following) as Lead_Source,
                       Last_Value (bizible2__form_url__c) Over (Partition By a.LeadId, a.MQL_Date ORDER BY convert_timezone('UTC', 'US/Pacific',c.bizible2__touchpoint_date__c) ASC rows between unbounded preceding and unbounded following) as Lead_Source_URL
                From newlead4 a
                Left Outer Join salesforce_bizible2__bizible_person__c b on a.leadid = b.bizible2__lead__c
                Left Outer Join salesforce_bizible2__bizible_touchpoint__c c on b.id = c.bizible2__bizible_person__c AND a.mql_date > convert_timezone('UTC', 'US/Pacific',c.bizible2__touchpoint_date__c)
                Where c.bizible2__Touchpoint_Position__c like '%Form%'
            ),

--9.6 Dedupe
newlead6 as (
                Select *
                From newlead5
                Group By leadid, mql_date, offer, touchpointid, touchpoint_date, lead_source, lead_source_url
            ),

--9.7 Adding NULL fields to 9.6 so that it can be joined with part8_final
newlead7 as (
                Select a.leadid, NULL as contactid, ISNULL(a.touchpoint_date, a.mql_date) as inquiry_date, a.mql_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as mql_reviewed_Date, NULL as mql_reviewed_value, NULL::timestamp AT TIME ZONE 'US/Pacific' as sal_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as lead_closed_date, NULL as lead_closed_value, NULL::timestamp AT TIME ZONE 'US/Pacific' as converted_date, NULL as opportunityid, NULL::timestamp AT TIME ZONE 'US/Pacific' as sql_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as sql_reviewed_Date, NULL as sql_reviewed_value, NULL::timestamp AT TIME ZONE 'US/Pacific' as discovery_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as sqo_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as opportunity_closed_date, NULL as opportunity_stage, a.offer, NULL::timestamp AT TIME ZONE 'US/Pacific' as prev_mql_date, a.touchpointid, a.touchpoint_date, a.lead_source, a.lead_source_url
                From newlead6 a
            ),
--241 secs

--9.8 The Union. This brings together all mqls being worked (part8_final) and all leads that have mql'd but haven't been worked
Part9_Final as (
                    Select * from newlead7
                    UNION ALL
                    Select * from part8_final
               ),
--790 secs

--------------------------------------------------------------------------------------------------------------------------------------
--Part 10: Inquiries
--------------------------------------------------------------------------------------------------------------------------------------  
--10.1 Identification of inquiries already in the model
Inquiry1 as (
                Select distinct touchpointid from part9_final where touchpointid IS NOT NULL
            ),

Inquiry2 as (
                Select a.*
                From salesforce_bizible2__bizible_touchpoint__c a
                Left Outer Join Inquiry1 b on a.Id = b.TouchpointId
                Where b.TouchpointId IS NULL
                AND a.bizible2__Touchpoint_Position__c like '%Form%'
             ),

Inquiry3 as (
                Select b.Bizible2__Lead__c,
                       b.Bizible2__Contact__c, 
                       convert_timezone('UTC', 'US/Pacific',a.bizible2__touchpoint_date__c) as Inquiry_Date, 
                       NULL::timestamp AT TIME ZONE 'US/Pacific' as MQL_Date, NULL::timestamp AT TIME ZONE 'US/Pacific' as mql_reviewed_Date, NULL as mql_reviewed_value, NULL::timestamp AT TIME ZONE 'US/Pacific' as sal_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as lead_closed_date, NULL as lead_closed_value, NULL::timestamp AT TIME ZONE 'US/Pacific' as converted_date, NULL as opportunityid, NULL::timestamp AT TIME ZONE 'US/Pacific' as sql_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as sql_reviewed_Date, NULL as sql_reviewed_value, NULL::timestamp AT TIME ZONE 'US/Pacific' as discovery_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as sqo_date, NULL::timestamp AT TIME ZONE 'US/Pacific' as opportunity_closed_date, NULL as opportunity_stage, NULL as offer, NULL::timestamp AT TIME ZONE 'US/Pacific' as prev_mql_date, 
                       a.id as touchpointid, 
                       convert_timezone('UTC', 'US/Pacific',a.bizible2__touchpoint_date__c) as touchpoint_date, 
                       a.bizible2__marketing_channel__c as lead_source, 
                       a.bizible2__form_url__c as lead_source_url
                From Inquiry2 a
                Inner Join salesforce_bizible2__bizible_person__c b on a.bizible2__bizible_person__c = b.id
            ),

Inquiry4 as (
                Select *
                From Inquiry3 a
                Where a.Bizible2__Contact__c <> ''
            ),

Inquiry5 as (
                Select b.Id as LeadId,
                       a.bizible2__contact__c as contactid, a.inquiry_date, a.mql_Date, a.mql_reviewed_date, a.mql_reviewed_value, a.sal_date, a.lead_closed_date, a.lead_closed_value, a.converted_date, a.opportunityid, a.sql_date, a.sql_reviewed_date, a.sql_reviewed_value, a.discovery_date, a.sqo_date, a.opportunity_closed_date, a.opportunity_stage, a.offer, a.prev_mql_date, a.touchpointid, a.touchpoint_date, a.lead_source, a.lead_source_url
                From Inquiry4 a
                Left Outer Join salesforce_lead b on a.bizible2__contact__c = b.ConvertedContactId 
            ),

Inquiry6 as (
                Select *
                From Inquiry3 a
                Where a.Bizible2__Contact__c = ''
            ),
--663 secs                  

Inquiry7 as (
                Select * From Inquiry6
                UNION ALL
                Select * From Inquiry5
            ),

Inquiry8 as (
                Select * from Inquiry7
                UNION ALL
                Select * from Part9_Final
            ),

Inquiry9 as (
              Select a.*, CASE WHEN b.bizible2__touchpoint_position__c like '%LC%' THEN 1 ELSE NULL END as Net_New_Name
              From Inquiry8 a
              Left Outer Join (Select * from salesforce_bizible2__bizible_touchpoint__c Where bizible2__Touchpoint_Position__c like '%Form%')b on a.touchpointid = b.id
            )

Select * From Inquiry9

2 个答案:

答案 0 :(得分:0)

Redshift可以优化CTE,但这是一个非常大的数字。

我建议将CTE子查询转换为临时表。

答案 1 :(得分:0)

此查询导致群集重启吗?请检查以下查询,

SELECT sysdate current_ts, endtime AS restart_ts
FROM stl_utilitytext
WHERE text LIKE '%xen_is_up.sql%'
ORDER BY endtime DESC;

你的群集是dc1.large吗?对于长查询,Redshift需要更多内存来重写它,它可能导致内存不足从而触发群集重启。

正如Joe所说,请尝试使用临时表而不是CTE。