Set-AzSynapseDataset : Object reference not set to an instance of an object.

[ClintHuijbers]

Description

I'm using the Az.Synapse modules within an Azure DevOps Pipeline (Azure PowerShell task version 5.*) to deploy triggers, pipelines and datasets.
While deleting and deploying triggers and pipelines runs without any problems.
The same code (syntax) raises errors for deploying datasets, while deleting datasets runs fine.

Error: Object reference not set to an instance of an object.

• CategoryInfo : CloseError: (:) [Set-AzSynapseDataset], NullReferenceException
• FullyQualifiedErrorId : Microsoft.Azure.Commands.Synapse.SetAzureSynapseDataset

I'm able to read the JSON file using Get-Content()

Here's a code example:

$FilePath = "D:\a\r1\a\Synapse-Build\DWHObjects\dataset\DIM_Calendar.json"
Get-Content $FilePath | ConvertFrom-Json | ConvertTo-Json
$result = Set-AzSynapseDataset -WorkspaceName $WorkspaceName -Name "DIM_Calendar" -DefinitionFile $FilePath

Issue script & Debug output

-

Environment data

-
Name                           Value                                                                                   
----                           -----                                                                                   
PSVersion                      5.1.20348.859                                                                           
PSEdition                      Desktop                                                                                 
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                 
BuildVersion                   10.0.20348.859                                                                          
CLRVersion                     4.0.30319.42000                                                                         
WSManStackVersion              3.0                                                                                     
PSRemotingProtocolVersion      2.3                                                                                     
SerializationVersion           1.1.0.1  


### Module versions

```PowerShell
Azure DevOps Pipeline (Azure PowerShell task version 5.*)


Name              : Az.Synapse
Path              : C:\Modules\az_7.5.0\Az.Synapse\1.3.0\Az.Synapse.psd1
Description       : Microsoft Azure PowerShell - Azure Synapse Analytics in Windows PowerShell and PowerShell Core.


### Error output

Set-AzSynapseDataset : Object reference not set to an instance of an object.
At D:\a\r1\a\Synapse-Build\DWHObjects\AzSynapseDeployDIMCalendar.ps1:85 char:11
+ $result = Set-AzSynapseDataset -WorkspaceName $WorkspaceName -Name "D ...
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Set-AzSynapseDataset], NullReferenceException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Synapse.SetAzureSynapseDataset
 
WARNING: Upcoming breaking changes in the cmdlet 'Resolve-AzError' :

The `Resolve-Error` alias will be removed in a future release.  Please change any scripts that use this alias to use 
`Resolve-AzError` instead.

Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other 
information on breaking changes in Azure PowerShell.

InnerException   : False
Exception        : System.NullReferenceException: Object reference not set to an instance of an object.
                      at Azure.Analytics.Synapse.Artifacts.Models.Dataset.Azure.Core.IUtf8JsonSerializable.Write(Utf8Js
                   onWriter writer)
                      at Azure.Analytics.Synapse.Artifacts.Models.DatasetResource.Azure.Core.IUtf8JsonSerializable.Writ
                   e(Utf8JsonWriter writer)
                      at Azure.Analytics.Synapse.Artifacts.DatasetRestClient.CreateCreateOrUpdateDatasetRequest(String 
                   datasetName, DatasetResource dataset, String ifMatch)
                      at Azure.Analytics.Synapse.Artifacts.DatasetRestClient.CreateOrUpdateDataset(String datasetName, 
                   DatasetResource dataset, String ifMatch, CancellationToken cancellationToken)
                      at Azure.Analytics.Synapse.Artifacts.DatasetClient.StartCreateOrUpdateDataset(String 
                   datasetName, DatasetResource dataset, String ifMatch, CancellationToken cancellationToken)
                      at Microsoft.Azure.Commands.Synapse.Models.SynapseAnalyticsArtifactsClient.CreateOrUpdateDataset(
                   String datasetName, String rawJsonContent)
                      at Microsoft.Azure.Commands.Synapse.SetAzureSynapseDataset.ExecuteCmdlet()
                      at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Message          : Object reference not set to an instance of an object.
StackTrace       :    at Azure.Analytics.Synapse.Artifacts.Models.Dataset.Azure.Core.IUtf8JsonSerializable.Write(Utf8Js
                   onWriter writer)
                      at Azure.Analytics.Synapse.Artifacts.Models.DatasetResource.Azure.Core.IUtf8JsonSerializable.Writ
                   e(Utf8JsonWriter writer)
                      at Azure.Analytics.Synapse.Artifacts.DatasetRestClient.CreateCreateOrUpdateDatasetRequest(String 
                   datasetName, DatasetResource dataset, String ifMatch)
                      at Azure.Analytics.Synapse.Artifacts.DatasetRestClient.CreateOrUpdateDataset(String datasetName, 
                   DatasetResource dataset, String ifMatch, CancellationToken cancellationToken)
                      at Azure.Analytics.Synapse.Artifacts.DatasetClient.StartCreateOrUpdateDataset(String 
                   datasetName, DatasetResource dataset, String ifMatch, CancellationToken cancellationToken)
                      at Microsoft.Azure.Commands.Synapse.Models.SynapseAnalyticsArtifactsClient.CreateOrUpdateDataset(
                   String datasetName, String rawJsonContent)
                      at Microsoft.Azure.Commands.Synapse.SetAzureSynapseDataset.ExecuteCmdlet()
                      at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
HelpLink         : 
ErrorDetails     : 
ErrorCategory    : CloseError: (:) [Set-AzSynapseDataset], NullReferenceException
InvocationInfo   : System.Management.Automation.InvocationInfo
ScriptStackTrace : at <ScriptBlock>, D:\a\r1\a\MonkeyDWH-Synapse-Build\DWHObjects\MonkeyDWHSynapseWorkspace\AzSynapseDe
                   ployDIMCalendar.ps1: line 85
                   at <ScriptBlock>, D:\a\_temp\38c63e63-679d-4821-9697-00dbb79efe9a.ps1: line 47
                   at <ScriptBlock>, <No file>: line 1


The Azure PowerShell team is listening, please let us know how we are doing: https://aka.ms/azpssurvey?Q_CHL=ERROR.

Thank you for your feedback. This has been routed to the support team for assistance.

[SaurabhSharma-MSFT]

@ClintHuijbers We are looking into it and get back to you for any additional information.

[ClintHuijbers]

Also in regards to Notebooks unfortunately:

Set-AzSynapseNotebook : Object reference not set to an instance of an object.

[SaurabhSharma-MSFT]

@ClintHuijbers I have tried creating a dataset using Set-AzSynapseDataset and it worked fine.

Can you please tell me the type of dataset you are trying to create? Also, if possible, can you share the dataset json so that I can try from my end as well.
Thanks

[ClintHuijbers]

I'm deploying a dataset which uses a Dedicated SQL Pool, which is also online while deploying.
More info:

Encoding:
UTF8

{
"name": "DIM_Calendar",
"properties": {
"annotations": [

                                   ],
                   "folder":  {
                                  "name":  "Data Mart/Dimensions/Sink"
                              },
                   "type":  "SqlPoolTable",
                   "schema":  [
                                  "@{name=Id; type=int}",
                                  "@{name=Date; type=date}",
                                  "@{name=DateTimestamp; type=datetime2; precision=3}",
                                  "@{name=MonthTimestamp; type=datetime2; precision=3}",
                                  "@{name=YearTimestamp; type=datetime2; precision=3}",
                                  "@{name=YearNumber; type=int}",
                                  "@{name=HalfYearNumber; type=int}",
                                  "@{name=QuarterNumber; type=int}",
                                  "@{name=MonthNumber; type=int}",
                                  "@{name=YearMonthNr; type=int}",
                                  "@{name=YearMonth; type=varchar}",
                                  "@{name=WeekNumber; type=int}",
                                  "@{name=YearWeek; type=varchar}",
                                  "@{name=ISOYearNumber; type=int}",
                                  "@{name=ISOWeekNumber; type=int}",
                                  "@{name=ISOYearWeek; type=varchar}",
                                  "@{name=MonthNumberOfQuarter; type=int}",
                                  "@{name=WeekNumberOfQuarter; type=int}",
                                  "@{name=WeekNumberOfMonth; type=int}",
                                  "@{name=DayNumberOfYear; type=int}",
                                  "@{name=DayNumberOfQuarter; type=int}",
                                  "@{name=DayNumberOfMonth; type=int}",
                                  "@{name=DayNumberOfWeek; type=int}",
                                  "@{name=IsLeap; type=int}",
                                  "@{name=IsWeekend; type=int}",
                                  "@{name=EnglishMonthName; type=varchar}",
                                  "@{name=EnglishMonthNameAbbreviation; type=varchar}",
                                  "@{name=EnglishDayName; type=varchar}",
                                  "@{name=EnglishDayNameAbbreviation; type=varchar}",
                                  "@{name=EnglishDateName; type=varchar}",
                                  "@{name=DutchMonthName; type=varchar}",
                                  "@{name=DutchMonthNameAbbreviation; type=varchar}",
                                  "@{name=DutchDayName; type=varchar}",
                                  "@{name=DutchDayNameAbbreviation; type=varchar}",
                                  "@{name=DutchDateName; type=varchar}"
                              ],
                   "sqlPool":  {
                                   "referenceName":  "devdedicatedsqlpool",
                                   "type":  "SqlPoolReference"
                               },
                   "typeProperties":  {
                                          "schema":  "datamart",
                                          "table":  "DIM_Calendar"
                                      }
               },
"type":  "Microsoft.Synapse/workspaces/datasets"

}

Deploying dataset #1:
Set-AzSynapseDataset : Object reference not set to an instance of an object.

• $result = Set-AzSynapseDataset -WorkspaceName $WorkspaceName -Name "D ...

•       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : CloseError: (:) [Set-AzSynapseDataset], NullReferenceException
  • FullyQualifiedErrorId : Microsoft.Azure.Commands.Synapse.SetAzureSynapseDataset