Monday, February 17, 2014

Populate varchar(max) or nvarchar(max) to SSIS string variable

How to import varchar(max) into SSIS string variable.

Database table

clip_image002

SSIS Varaibles
clip_image004

SSIS query result set object
clip_image006

The foreach loop has to have something in it otherwise the variables don’t get populated.

clip_image008

Execute SQL Task set up. There is only one record returned from the query. Make sure the the SQLStatement handles NULLs via ISNULL(xxx,'') or COALESCE(xxx,'')

clip_image010

clip_image012

Foreach loop setup.

clip_image014

clip_image016

Wednesday, November 28, 2012

MDS - Operation is not valid due to the current state of the objec


If you have a large model and you are trying edit security settings in MDS(Master Data Services) you may get the following error:

An unknown error occurred.

or
Operation is not valid due to the current state of the object

or
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: Operation is not valid due to the current state of the object.

Source Error: 
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace: 
[InvalidOperationException: Operation is not valid due to the current state of the object.]
   System.Web.HttpValueCollection.FillFromEncodedBytes(Byte[] bytes, Encoding encoding) +11371791
   System.Web.HttpRequest.FillInFormCollection() +329

[HttpException (0x80004005): The URL-encoded form data is not valid.]
   System.Web.HttpRequest.FillInFormCollection() +11485906
   System.Web.HttpRequest.get_Form() +157
   System.Web.HttpRequest.get_HasForm() +11486708
   System.Web.UI.Page.GetCollectionBasedOnMethod(Boolean dontReturnNull) +141
   System.Web.UI.Page.DeterminePostBackMode() +100
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +259

Solution
Thanks to the MS MDS support team you need to add the following to the web.config file located C:\Program Files\Microsoft SQL Server\110\Master Data Services\WebApplication

<configuration>
<appSettings>
<add key="aspnet:MaxHttpCollectionKeys" value="2000" />
</appSettings>
</configuration>

appSettings is located near the end of the file. If the error is still occurring increase the value 10000

Wednesday, November 14, 2012

MDS Missing attribute


Tag
Master Data Services

I had an issue where the subscription view would show all the attributes but when trying to edit the entity in Explorer or System Administration the attribute would not appear.
So started looking at mdm.tblAttribute table and there it was. So then I looked in mdm.viw_SYSTEM_SCHEMA_ATTRIBUTES and it was missing


SELECT
*
FROM
mdm.tblModel tMod  
JOIN mdm.tblEntity tEnt ON tMod.ID = tEnt.Model_ID  
JOIN mdm.tblAttribute tAtt ON tEnt.ID = tAtt.Entity_ID  
JOIN  mdm.tblEntityMemberType tMemberType ON tAtt.MemberType_ID = tMemberType.ID
JOIN mdm.tblList tAttributeType ON tAttributeType.ListCode = N'lstAttributeType' AND tAtt.AttributeType_ID = tAttributeType.OptionID
JOIN mdm.tblList tDataType ON tDataType.ListCode = N'lstDataType' AND tAtt.DataType_ID = tDataType.OptionID
--JOIN mdm.tblUser usrE ON tAtt.EnterUserID = usrE.ID
--JOIN mdm.tblUser usrL ON tAtt.LastChgUserID = usrL.ID
LEFT OUTER JOIN mdm.tblList tDataMask ON tDataMask.ListCode = N'lstInputMask' AND tAtt.InputMask_ID = tDataMask.OptionID AND tAtt.DataType_ID = tDataMask.Group_ID
LEFT OUTER JOIN mdm.tblEntity tDBAEnt ON tAtt.DomainEntity_ID = tDBAEnt.ID

It turns out I had DELETED all the users from the mdm.tblUser table except for user ID 1. Problem is viw_SYSTEM_SCHEMA_ATTRIBUTES does an INNER JOIN with the tblUser table.

Run the following to check if there are any issues.


SELECT * FROM mdm.tblEntity WHERE
EnterUserID <> 1
OR LastChgUserID <> 1

SELECT * FROM mdm.tblAttribute WHERE
EnterUserID <> 1
OR LastChgUserID <> 1

UPDATE mdm.tblAttribute SET EnterUserID = 1, LastChgUserID = 1 WHERE EnterUserID <> 1 OR LastChgUserID <> 1


Thursday, October 11, 2012

MDS Master Data Services 2012 Rename Entity Staging Table


DECLARE @StagingTable NVARCHAR(100) = 'StagingTableOldName'
DECLARE @NewStagingTable NVARCHAR(100) = 'StagingTableNewName'
DECLARE @EntityID INT

DECLARE db_cursor CURSOR FOR
SELECT e.ID, e.StagingBase, @NewStagingTable
  FROM [mdm].[tblEntity] e
  INNER JOIN mdm.tblModel m ON e.Model_ID = m.ID
  WHERE m.Name = 'MTN'
  AND e.StagingBase = @StagingTable
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @EntityID, @StagingTable, @NewStagingTable
WHILE @@FETCH_STATUS = 0
BEGIN
PRINT '---- ' + @StagingTable
PRINT 'UPDATE mdm.tblEntity SET StagingBase = ' + QUOTENAME(@NewStagingTable, '''') + ' WHERE ID = ' + CAST(@EntityID AS NVARCHAR(10))
PRINT 'DROP PROC stg.udp_' + @StagingTable + '_Leaf'
PRINT 'DROP VIEW stg.viw_' + @StagingTable + '_MemberErrorDetails'
 
PRINT 'EXEC sp_rename ' + QUOTENAME('stg.' + @StagingTable + '_Leaf', '''') + ', ' + QUOTENAME(@NewStagingTable + '_Leaf', '''')
PRINT 'EXEC sp_rename ' + QUOTENAME('stg.' + @NewStagingTable + '_Leaf.pk_' + @StagingTable+ '_Leaf', '''') + ', ' + QUOTENAME('pk_' + @NewStagingTable + '_Leaf', '''')
PRINT 'EXEC sp_rename ' + QUOTENAME('stg.ck_' + @StagingTable + '_Leaf_ImportType', '''') + ', ' + QUOTENAME('ck_' + @NewStagingTable + '_Leaf_ImportType', '''')
PRINT 'EXEC sp_rename ' + QUOTENAME('stg.ck_' + @StagingTable + '_Leaf_ImportStatus_ID', '''') + ', ' + QUOTENAME('ck_' + @NewStagingTable + '_Leaf_ImportStatus_ID', '''')
PRINT 'EXEC sp_rename ' + QUOTENAME('stg.' + @NewStagingTable + '_Leaf.ix_' + @StagingTable + '_Leaf_Batch_ID', '''') + ', ' + QUOTENAME('ix_' + @NewStagingTable + '_Leaf_Batch_ID', '''')
PRINT 'EXEC sp_rename ' + QUOTENAME('stg.' + @NewStagingTable + '_Leaf.ix_' + @StagingTable + '_Leaf_BatchTag', '''') + ', ' + QUOTENAME('ix_' + @NewStagingTable + '_Leaf_BatchTag', '''')

PRINT 'EXEC sp_rename ' + QUOTENAME('stg.df_' + @StagingTable + '_Leaf_ImportStatus_ID', '''') + ', ' + QUOTENAME('df_' + @NewStagingTable + '_Leaf_ImportStatus_ID', '''')
PRINT 'EXEC mdm.udpEntityStagingCreateLeafStoredProcedure ' + CAST(@EntityID AS NVARCHAR(10))
PRINT 'EXEC mdm.udpCreateEntityStagingErrorDetailViews ' + CAST(@EntityID AS NVARCHAR(10))
PRINT ''
FETCH NEXT FROM db_cursor INTO @EntityID, @StagingTable, @NewStagingTable
END
CLOSE db_cursor
DEALLOCATE db_cursor

Thursday, August 9, 2012

MDS Checking Users Security Settings

Hi,

Here are some sql scripts that you can run to check the security settings of a user in MDS (Master Data Services).

-------------------------------------------------------------------------------      
-- Model User Security
-------------------------------------------------------------------------------
SELECT * FROM mdm.udfSecurityUserModelList(5) ACL
INNER JOIN mdm.viw_SYSTEM_SCHEMA_MODEL vModel ON ACL.ID = vModel.ID
INNER JOIN mdm.tblSecurityPrivilege tPriv ON tPriv.ID = acl.Privilege_ID
-------------------------------------------------------------------------------      
-- Entity User Security
-------------------------------------------------------------------------------
SELECT vEnt.Model_ID,vEnt.Model_Name,
acl.ID AS 'Entity_ID', vEnt.Name AS 'Entity_Name', tPriv.Code
FROM    mdm.udfSecurityUserEntityList(5, NULL) acl
INNER JOIN mdm.viw_SYSTEM_SCHEMA_ENTITY vEnt ON  acl.ID = vEnt.ID 
INNER JOIN mdm.tblSecurityPrivilege tPriv ON tPriv.ID = acl.Privilege_ID
ORDER BY vEnt.Model_ID, vEnt.Name
-------------------------------------------------------------------------------      
-- Attribute Group User Security
-------------------------------------------------------------------------------
SELECT * 
FROM    mdm.udfSecurityUserAttributeGroupList(5, 4,149,1) acl
INNER JOIN mdm.viw_SYSTEM_SCHEMA_ATTRIBUTEGROUPS vAG ON  acl.ID = vAG.ID 
INNER JOIN mdm.tblSecurityPrivilege tPriv ON tPriv.ID = acl.Privilege_ID
ORDER BY vAG.Model_ID, vAG.Name
-------------------------------------------------------------------------------      
-- Attribute Group Attribute List User Security
-------------------------------------------------------------------------------
SELECT acl.USER_ID, acl.Model_ID, vAG.Model_Name, acl.Entity_ID,
vAG.Entity_Name, acl.Privilege_ID, tPriv.Code AS 'SecurityAttributeGroup_Code',
vAG.ID AS 'AttributeGroup_ID', vAG.Name AS 'AttributeGroup_Name',
vAGA.Attribute_ID,vAGA.Attribute_Name, tPrivA.Code AS 'SecurityAttribute_Code'
FROM mdm.udfSecurityUserAttributeGroupList(1, 4,149,1) acl
INNER JOIN mdm.tblSecurityPrivilege tPriv ON tPriv.ID = acl.Privilege_ID
INNER JOIN mdm.viw_SYSTEM_SCHEMA_ATTRIBUTEGROUPS vAG ON  acl.ID = vAG.ID 
INNER JOIN  mdm.viw_SYSTEM_SCHEMA_ATTRIBUTEGROUPS_ATTRIBUTES vAGA ON vAG.ID = vAGA.AttributeGroup_ID
INNER JOIN mdm.udfSecurityUserAttributeList(1, 4,149,1) acla ON acla.ID = vAGA.Attribute_ID
INNER JOIN mdm.tblSecurityPrivilege tPrivA ON tPrivA.ID = acla.Privilege_ID
ORDER BY vAG.Model_ID, vAG.Name

-------------------------------------------------------------------------------      
-- Attribute List User Security
-------------------------------------------------------------------------------
SELECT va.Model_ID, va.Model_Name, va.Entity_ID, va.Entity_Name,
va.Attribute_ID, va.Attribute_Name, tPrivA.Code AS 'Security_Code'
FROM mdm.udfSecurityUserAttributeList(5, 4,149,1) acla
INNER JOIN mdm.viw_SYSTEM_SCHEMA_ATTRIBUTES vA ON acla.ID = vA.Attribute_ID
INNER JOIN mdm.tblSecurityPrivilege tPrivA ON tPrivA.ID = acla.Privilege_ID
ORDER BY vA.Model_ID, vA.Entity_Name, vA.Attribute_Name

Saturday, May 26, 2012

MDS .net Assembly Error "The Microsoft SQL Server license has expired"


I was trying to get MDS assemblies going.

File Program.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Collections.ObjectModel;
using Microsoft.MasterDataServices.Deployment;
using Microsoft.MasterDataServices.Services.DataContracts;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                //Reads a model's metadata, business rules, and master data
                ModelReader reader = new ModelReader();

                Console.WriteLine(reader.CurrentUser.DisplayName);
                Console.WriteLine("Models:");
                Collection<identifier> models = reader.GetModels();
                foreach (Identifier modelId in models)
                {
                    Console.WriteLine(modelId.Name);
                }
            }
            catch(System.Exception ex)
            {
                Console.WriteLine("Error: " + ex.Message);

            }
            Console.ReadKey();
        }
    }
}

App.Config

<configuration>
    <configsections>
    </configsections>
    <connectionstrings>
        <add connectionstring="Data Source=localhost;Initial Catalog=MDS_2012;Integrated Security=True" name="defaultMdsConnection" providername="System.Data.SqlClient">
    </add></connectionstrings>
</configuration>

referencing the following assemblies from
C:\Program Files\Microsoft SQL Server\110\Master Data Services\WebApplication\bin
Microsoft.MasterDataServices.Core
Microsoft.MasterDataServices.Deployment
Microsoft.MasterDataServices.Services
Microsoft.MasterDataServices.Services.Contracts

When running the app I get the following error:
Olof Szymczak
Models:
Error: The Microsoft SQL Server license has expired.


When using reflector, I find that the following method throws the error
File: Services.cs
Namespace: namespace Microsoft.MasterDataServices.Services
Method:
private static void CheckExpiration()
        {
            if (isExpired)
            {
                Microsoft.MasterDataServices.Core.Log.Write(LogMessageType.Error, "Evaluation period has expired.");
                EditionExpiredMessage detail = new EditionExpiredMessage();
                FaultException<editionexpiredmessage> exception = new FaultException<editionexpiredmessage>(detail, ErrorStrings.EvaluationExpired);
                throw exception;
            }
        }

It looks like isExpired is set to true. So I looked for the method that is setting isExpired. Drilling in that method I find:
File: Utility.cs
Namespace: Microsoft.MasterDataServices.Core.BusinessEntities
Method: private static QuerySkuValue LoadQuerySkuValueMethod()
in the method it is trying to load a dll C:\Program Files\Microsoft SQL Server\110\Shared\sqlboot.dll
                    zero = LoadLibraryW(dllname); //this fails to load the dll

Checked the location and the dll exists.

Note if I set isExpired = false while debugging the code. I get the correct results.

Eventually I found the answer after downloading http://archive.msdn.microsoft.com/ModelDApiSample, modifing it to point to the MDS 2012 assemblies and reconfiguring the config file. I run it and it worked, I was going WTF. So I checked what was wrong with my project settings, the answer x86, the project.Build.Platform target has to be set to either x64 or Any CPU.

All I can say is what a great way of telling you, that you have compiled the code for the wrong platform by returning an error of "The Microsoft SQL Server license has expired".

Friday, April 27, 2012

Install MDS on the Extranet

Hi,
I had to install MDS portal on the extranet (DMZ) and MDS database on the intranet. Here are some of the issues that I encountered.

Setup

DomainA (Intranet/Internal) ServerA SqlServer 2008 R2/2012
Firewall Firewall rule to let sql connection through. No Federated Domain
DomainB (Extranet/DMZ) ServerB IIS 7.5 make sure that IIS and WCF is installed correctly

Creating the database

Since I can't use Current User - Integrated Security I have to switch to SQL Server Account. Make sure that you create a sql login user with sufficient rights to create the database.

Creating Website

Everything works fine until you try to connect the database to the portal. You will get the following error:
Service Account Failure User or Role '(null)' does not exist in this database.
No problem you have to go the web.config file and add the settings manually. In this example I have added 2 sites dev and test
...
...    <masterDataServices>
        <instance virtualPath="" siteName="MDS_MTN_DEV" connectionName="MDS1" serviceName="MDS1" />
        <instance virtualPath="" siteName="MDS_MTN_TEST" connectionName="MDS2" serviceName="MDS2" />
    </masterDataServices>
    <connectionStrings>
        <add name="MDS1" connectionString="Data Source=[ServerNameOrServerIP];Initial Catalog=mds;User Id=[username];Password=[password];Connect Timeout=60" />
        <add name="MDS2" connectionString="Data Source=[ServerNameOrServerIP];Initial Catalog=mds_test;User Id=[username];Password=[password];Connect Timeout=60" />
    </connectionStrings>
...
...    
    <service behaviorConfiguration="mdsWsHttpBehavior" name="MDS1">
        <endpoint binding="wsHttpBinding" bindingConfiguration="mdsWsHttpBinding"
         bindingNamespace="http://schemas.microsoft.com/sqlserver/masterdataservices/2009/09"
         contract="Microsoft.MasterDataServices.Services.ServiceContracts.IService" />
        <endpoint address="bhb" binding="basicHttpBinding" bindingConfiguration="mdsBasicHttpBinding"
         bindingNamespace="http://schemas.microsoft.com/sqlserver/masterdataservices/2009/09"
         contract="Microsoft.MasterDataServices.Services.ServiceContracts.IService" />
       </service>
       <service behaviorConfiguration="mdsWsHttpBehavior" name="MDS2">
        <endpoint binding="wsHttpBinding" bindingConfiguration="mdsWsHttpBinding"
         bindingNamespace="http://schemas.microsoft.com/sqlserver/masterdataservices/2009/09"
         contract="Microsoft.MasterDataServices.Services.ServiceContracts.IService" />
        <endpoint address="bhb" binding="basicHttpBinding" bindingConfiguration="mdsBasicHttpBinding"
         bindingNamespace="http://schemas.microsoft.com/sqlserver/masterdataservices/2009/09"
         contract="Microsoft.MasterDataServices.Services.ServiceContracts.IService" />
       </service>
...
...
If you close and reopen Master Data Services Configuration Manager and go the website you will see that the website is now connected to the database.

Testing the MDS Portal

When viewing an entity in explorer you may get the following errors:
IE8 Error Message [HttpWebRequest_WebException_RemoteServer] Arguments: NotFound Debugging resource strings are unavaialble. Often the key and argumetns provide sufficient information to diagnose the problem. See http://go.microsoft.com/fwlink/linkid=1066638&Version5.0.61118.08&File=System.Windows.dll&Key=HttpWebRequest_WebException_RemoteServer
IE9 The remote server returned an error: NotFound.

Testing the MDS Service

Check Authorization on the Portal and Service in IIS Portal – Windows Authentication Enabled Service – Anonymous Authentication Enabled and Windows Authentication Enabled You may need to do an IISRESET. http://localhost/service/service.svc
Error Message for Anonymous Authentication not enabled Security settings for this service require 'Anonymous' Authentication but it is not enabled for the IIS application that hosts this service.  Error Message for Windows Authentication not enabled Security settings for this service require Windows Authentication but it is not enabled for the IIS application that hosts this service.
Error Message: The HttpsGetEnabled property of ServiceMetadataBehavior is set to true and the HttpsGetUrl property is a relative address, but there is no https base address.  Either supply an https base address or set HttpsGetUrl to an absolute address.
Change the web.config file to the following if only using http httpsGetEnabled="false"
<serviceMetadata httpGetEnabled="true" httpsGetEnabled="false"/>
After all that it works!!!