{"id":2154,"date":"2018-08-08T20:36:16","date_gmt":"2018-08-08T18:36:16","guid":{"rendered":"http:\/\/miroslaw.borodziuk.eu\/?p=2154"},"modified":"2018-08-19T12:11:52","modified_gmt":"2018-08-19T10:11:52","slug":"implementacja-magazynu-danych-w-azure","status":"publish","type":"post","link":"http:\/\/miro.borodziuk.eu\/index.php\/2018\/08\/08\/implementacja-magazynu-danych-w-azure\/","title":{"rendered":"Azure Blob storage"},"content":{"rendered":"<p>Poj\u0119cie <em>blob<\/em> to skr\u00f3t od<em> binary large objects<\/em>. Azure Blob storage to us\u0142uga u\u017cywana do sk\u0142adowania du\u017cych ilo\u015bci nieustrukturyzowanych danych takich jak dokumenty, pliku multimedialne, backupy.<\/p>\n<p><!--more--><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: #3366ff;\">Identify appropriate blob type for specific storage requirements<\/span><\/p>\n<p>Istniej\u0105 dwa typy blobs:<\/p>\n<ul>\n<li><em>Block blobs<\/em> &#8211; typ zaprojektowany do efektywnego uploadu danych. Przeznaczony do sk\u0142adowania plik\u00f3w takich jak wideo, obrazki i dokumenty. Ka\u017cdy blob sk\u0142ada si\u0119 z blok\u00f3w, kt\u00f3re s\u0105 identyfikowane przez ID. Maksymalny rozmiar bloku to 200GB. Jedyna us\u0142uga storage, kt\u00f3ra wspiera ZRS.<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2247 size-full\" src=\"http:\/\/miroslaw.borodziuk.eu\/wp-content\/uploads\/BlockBlobs.png\" alt=\"\" width=\"605\" height=\"275\" srcset=\"http:\/\/miro.borodziuk.eu\/wp-content\/uploads\/BlockBlobs.png 605w, http:\/\/miro.borodziuk.eu\/wp-content\/uploads\/BlockBlobs-300x136.png 300w\" sizes=\"(max-width: 605px) 100vw, 605px\" \/><\/li>\n<li><em>Append blobs<\/em> &#8211; sk\u0142adaj\u0105 si\u0119 z blok\u00f3w (blocks) i s\u0105 optymalizowane do operacji dodawania (append). Modyfikacja blob rodzaju append polega na do\u0142o\u017ceniu na ko\u0144cu bloba kolejnych blok\u00f3w. Kasowanie lub aktualizowanie istniej\u0105cego append bloba nie jest mo\u017cliwe.<\/li>\n<li><em>Page blobs<\/em> &#8211; nazywane tak\u017ce dyskami poniewa\u017c s\u0105 optymalizowane pod k\u0105tem losowego zapisu i mog\u0105 mie\u0107 pojemno\u015b\u0107 do 8 TB. Wirtualne twarde dyski w Azure (VHDs) s\u0105 sk\u0142adowane jako page blob.<img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2248 size-full\" src=\"http:\/\/miroslaw.borodziuk.eu\/wp-content\/uploads\/PageBlobs.png\" alt=\"\" width=\"568\" height=\"239\" srcset=\"http:\/\/miro.borodziuk.eu\/wp-content\/uploads\/PageBlobs.png 568w, http:\/\/miro.borodziuk.eu\/wp-content\/uploads\/PageBlobs-300x126.png 300w\" sizes=\"(max-width: 568px) 100vw, 568px\" \/><\/li>\n<\/ul>\n<p>Wi\u0119cej o rodzajach blobs:<\/p>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/rest\/api\/storageservices\/understanding-block-blobs--append-blobs--and-page-blobs\">https:\/\/docs.microsoft.com\/en-us\/rest\/api\/storageservices\/understanding-block-blobs&#8211;append-blobs&#8211;and-page-blobs<\/a><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: #3366ff;\">Configure Blob-Level Tiering (Hot, Cool, Archive)<\/span><\/p>\n<p>Istniej\u0105 trzy poziomy dost\u0119pu do blobs r\u00f3\u017cni\u0105ce si\u0119 cen\u0105:<\/p>\n<ul>\n<li><em>Hot<\/em> &#8211; przewidziane dla danych, do kt\u00f3rych wymagany jest cz\u0119sty dost\u0119p. Wy\u017cszy ni\u017c cool i archive koszt sk\u0142adowania danych ale ni\u017cszy koszt dost\u0119pu do danych.<\/li>\n<li><em>Cool<\/em> &#8211; dane niezbyt cz\u0119sto u\u017cytkowane, sk\u0142adowane co najmniej 30 dni. Ni\u017cszy ni\u017c hot koszt sk\u0142adowania danych ale wy\u017cszy koszt dost\u0119pu. Np. kr\u00f3tkoterminowy backup.<\/li>\n<li><em>Archive<\/em> &#8211; dane rzadko u\u017cywane i sk\u0142adowane co najmniej 180 dni z czasem dost\u0119pu liczonym w godzinach (&lt; 15 godzin). Najni\u017cszy koszt sk\u0142adowania i najwy\u017cszy koszt dost\u0119pu do tych danych.<\/li>\n<\/ul>\n<p>Poziomy obs\u0142uguj\u0105 tylko konta storage typu GPv2 (General Purpose). GPv1 nie maj\u0105 takiej funkcjonalno\u015bci. Upgrade kont storage GPv1 do GPv2:<\/p>\n<pre class=\"lang:ps decode:true\">PS C:\\&gt; Set-AzureRmStorageAccount -ResourceGroupName &lt;resource-group&gt; -AccountName &lt;storage-account&gt; -UpgradeToStorageV2<\/pre>\n<p>Utworzenie storaga rodzaju BlobStorage i poziom dost\u0119pu Hot.<\/p>\n<pre class=\"lang:ps decode:true\">PS C:\\&gt; New-AzureRmStorageAccount -ResourceGroupName RG01 -AccountName mystrorageaccount001 -Location westeurope -SkuName Standard_GRS -Kind BlobStorage -AccessTier Hot<\/pre>\n<p>Zmiana poziomu dost\u0119pu na cool:<\/p>\n<pre class=\"lang:ps decode:true\">PS C:\\&gt;Set-AzureRmStorageAccount -ResourceGroupName \"MyResourceGroup\" -AccountName \"mystorageaccount\" -AccessTier Cool\r\nool<\/pre>\n<p>Wi\u0119cej o poziomach dost\u0119pu:<\/p>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/storage\/blobs\/storage-blob-storage-tiers\">https:\/\/docs.microsoft.com\/en-us\/azure\/storage\/blobs\/storage-blob-storage-tiers<\/a><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: #3366ff;\">Kontenery.<\/span><\/p>\n<p>Je\u017celi blob storage przyr\u00f3wnamy do koncepcji folder\u00f3w i plik\u00f3w to odpowiednikiem folderu b\u0119dzie kontener, a odpowiednikiem pliku b\u0119dzie blob.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2306 size-full\" src=\"http:\/\/miroslaw.borodziuk.eu\/wp-content\/uploads\/Azure-Container-1.png\" alt=\"\" width=\"469\" height=\"286\" srcset=\"http:\/\/miro.borodziuk.eu\/wp-content\/uploads\/Azure-Container-1.png 469w, http:\/\/miro.borodziuk.eu\/wp-content\/uploads\/Azure-Container-1-300x183.png 300w\" sizes=\"(max-width: 469px) 100vw, 469px\" \/><\/p>\n<p>W spos\u00f3b przedstawiony poni\u017cej mo\u017cna utworzy\u0107 kontener. Ustawienie Parametr <code>Permission <\/code>ustawia typ dost\u0119pu do kontenera:<\/p>\n<ul>\n<li><em>Off<\/em> &#8211; dost\u0119p jest dozwolony tylko do w\u0142a\u015bciciela konta storage. Domy\u015blny poziom dost\u0119pu. W portalu Azure ten tryb nazywa si\u0119 <em>Private<\/em>.<\/li>\n<li><em>Blob<\/em> &#8211; tryb tylko do odczytu do danych w kontenerze ale nie do samego kontenera.<\/li>\n<li><em>Container<\/em> &#8211; tryb tylko do odczytu do danych w kontenerze i do kontenera.<\/li>\n<\/ul>\n<pre class=\"lang:ps decode:true\">$ResourceGroup = $RG01\r\n$StorageAccountName = \"mystorageaccount0001\"\r\n$ContainerName=\"mycontainer01\"\r\n\r\n$StorageAccount=New-AzureRmStorageAccount -ResourceGroupName RG01 -AccountName $StorageAccountName -Location westeurope -SkuName Standard_GRS \r\n$StorageContext = $storageAccount.Context\r\nNew-AzureStorageContainer -Name $ContainerName -Context $StorageContext -Permission Off\r\n<\/pre>\n<p>Lub mo\u017cna przed za\u0142o\u017ceniem kontenera utworzy\u0107 kontekst. Wcze\u015bniej jednak potrzebujemy odczyta\u0107 podstawowy klucz dost\u0119pu do magazynu.<\/p>\n<pre class=\"lang:ps decode:true \">$ResourceGroup = \"RG01\"\r\n$StorageAccountName = \"mystorageaccount0002\"\r\n$ContainerName=\"mycontainer02\"\r\n\r\n$StorageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $ResourceGroup -AccountName $StorageAccountName).Value[0]\r\n$StorageContext = New-AzureStorageContext \u2013StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey\r\n\r\nNew-AzureStorageContainer -Name $ContainerName -Context $StorageContext -Permission Off<\/pre>\n<p>&nbsp;<\/p>\n<p><span style=\"color: #3366ff;\">Design blob hierarchies<\/span><\/p>\n<p>Us\u0142uga blob oparta jest na p\u0142askim schemacie co oznacza, \u017ce poni\u017cej poziomu root mo\u017cna utworzy\u0107 tylko jeden kontener. Kontener\u00f3w nie mo\u017cna zagnie\u017cd\u017ca\u0107 w innych kontenerach. P\u0142aski schemat Azure Blob storage wygl\u0105da nast\u0119puj\u0105co:<\/p>\n<p><code>Object structure:\u00a0\u00a0\u00a0 Storage account \u00bb Container \u00bb Blob name<\/code><\/p>\n<p><code>Example:\u00a0\u00a0\u00a0 sa \u00bb [blob.core.windows.net] \u00bb raw \u00bb picture1. rif<\/code><\/p>\n<p>Najcz\u0119\u015bciej hierarchi\u0119 magazynowania blobs realizuje si\u0119 przez dodawania odpowiednich prefiks\u00f3w do nazwy blobs.<\/p>\n<p><code>sa \u00bb [blob.core.windows.net] \u00bb raw \u00bb pictures\/picture1.rif<\/code><br \/>\n<code>sa \u00bb [blob.core.windows.net] \u00bb raw \u00bb audio\/recording1.pcm<\/code><\/p>\n<p><code>pictures\/picture1.rif<\/code> i<code> audio\/recording1.pcm<\/code> to nazwy blobs.<\/p>\n<p>Mo\u017cna to tak\u017ce osi\u0105gn\u0105\u0107 przed dodaniem prefiksu do nazwy kontenera. Nazwa kontenera nie mo\u017ce jednak zawiera\u0107 znaku <code>\/<\/code>.<\/p>\n<p><code>sa \u00bb [blob.core.windows.net] \u00bb raw-pictures \u00bb picture1.rif<\/code><br \/>\n<code>sa \u00bb [blob.core.windows.net] \u00bb raw-audio \u00bb recording1.rif<\/code><\/p>\n<p>Takie rozwi\u0105zanie upraszcza u\u017cywanie\u00a0 filtr\u00f3w\u00a0 podczas pobierania blobs\u00f3w. Ponad to mo\u017cna u\u017cywa\u0107 innych filtr\u00f3w bezpiecze\u0144stwa opartych o hierarchi\u0119.<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"color: #3366ff;\">Implement async blob copy<\/span><\/p>\n<p>To initiate a blob copy, use the Start-AzureStorageBlobCopy cmdlet. This cmdlet accepts either the source URI (if it is external), or as the example below shows, the blob name, container, and storage context to access the source blob in an Azure Storage account. The destination requires the container name, blob name, and a storage context for the destination storage account.<\/p>\n<pre class=\"lang:ps decode:true \">$blobCopyState = Start-AzureStorageBlobCopy -SrcBlob $vhdName `\r\n-SrcContainer $srcContainer `\r\n-Context $srcContext `\r\n-DestContainer $destContainer `\r\n-DestBlob $vhdName `\r\n-DestContext $destContext<\/pre>\n<p>Let\u2019s review the parameters in the preceding example:<\/p>\n<ul>\n<li>The SrcBlob parameter expects the file name of source file to start copying.<\/li>\n<li>The SrcContainer parameter is the container the source file resides in.<\/li>\n<li>The Context parameter accepts a context object created by the New-AzureStorageContext cmdlet. The context has the storage account name and key for the source storage account and is used for authentication.<\/li>\n<li>The DestContainer is the destination container to copy the blob to. The call will fail if this container does not exist on the destination storage account.<\/li>\n<li>The DestBlob parameter is the filename of the blob on the destination storage account. The destination blob name does not have to be the same as the source.<\/li>\n<li>The DestContext parameter also accepts a context object created with the details of the destination storage account including the authentication key<\/li>\n<\/ul>\n<p>MULTIPLE SUBSCRIPTIONS REQUIRES A CALL TO SELECT AZURESUBSCRIPTION<br \/>\nTo copy between storage accounts in separate subscriptions, you need to call SelectAzureSubscription between the calls to Get-AzureStorageKey to switch to the alternate subscription. Here is a complete example of how to use the Start-AzureStorageBlob copy cmdlet to copy a blob between two storage accounts.<\/p>\n<pre class=\"lang:ps decode:true\">$vhdName = \"[file name]\"\r\n$srcContainer = \"[source container]\"\r\n$destContainer = \"[destination container]\"\r\n$srcStorageAccount = \"[source storage]\"\r\n$destStorageAccount = \"[dest storage]\"\r\n$srcStorageKey = (Get-AzureStorageKey -StorageAccountName $srcStorageAccount).Primary\r\n$destStorageKey = (Get-AzureStorageKey -StorageAccountName $destStorageAccount).Primary\r\n$srcContext = New-AzureStorageContext \u2013StorageAccountName $srcStorageAccount -StorageAccountKey $srcStorageKey\r\n$destContext = New-AzureStorageContext \u2013StorageAccountName $destStorageAccount -StorageAccountKey $destStorageKey\r\n\r\nNew-AzureStorageContainer -Name $destContainer -Context $destContext\r\n$copiedBlob = Start-AzureStorageBlobCopy -SrcBlob $vhdName `\r\n-SrcContainer $srcContainer `\r\n-Context $srcContext `\r\n-DestContainer $destContainer `\r\n-DestBlob $vhdName `\r\n-DestContext $destContext<\/pre>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Poj\u0119cie blob to skr\u00f3t od binary large objects. Azure Blob storage to us\u0142uga u\u017cywana do sk\u0142adowania du\u017cych ilo\u015bci nieustrukturyzowanych danych takich jak dokumenty, pliku multimedialne, backupy.<\/p>\n","protected":false},"author":1,"featured_media":2224,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[61,70],"tags":[],"_links":{"self":[{"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/posts\/2154"}],"collection":[{"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/comments?post=2154"}],"version-history":[{"count":55,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/posts\/2154\/revisions"}],"predecessor-version":[{"id":2307,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/posts\/2154\/revisions\/2307"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/media\/2224"}],"wp:attachment":[{"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/media?parent=2154"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/categories?post=2154"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/miro.borodziuk.eu\/index.php\/wp-json\/wp\/v2\/tags?post=2154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}