questions about how to deploy published service on the other platform deployment?

[giggle0312]

I have followed the introduction of "scikit_bring_your_own" to run the example, but I encounter some questions .
1、If I want to utilize the resource of AWS, I will push the docker image to sagemaker and excute the processing including "creating training Job"。After training job, I can find the s3 path of trained model named"model.tar.gz" and ec2 path of docker image on the console, can I download this model.tar.gz and docker image and deploy them on the other platform which has no relationship with AWS to make prediction? If can, can you give me an example?

2、If I trained the model with local mode, and then I put the trained model nameed "decision-tree.pkl" in my docker container. And After I push the docker image to sagemaker, can I run "creating new model" directly without creating new trainng job. And then deploy on sagemaker to publish a webservice.

[giggle0312]

I have another question
The trained model is Serialized by mxnet format. If I install apache mxnet package on my local environment without sagemaker, I can deserialized the model and predict my test data?

[djarpin]

Hi @giggle0312 .

1. Yes. The model artifact is yours, so you can host it for predictions outside of Amazon SageMaker. How you would do this probably depends on the specific setup of the non-SageMaker platform. If the other platform supports containers, then you might just be able to pick up the container as-is and host there. If it requires Python code, then you might just be able to snippets from predictor.py. If it requires a PMML object, then you'll have to untar model.tar.gz and convert the .pkl to PMML manually. Etc. The large number of potential variations on what hosting in another platform might look like is one of the reasons we don't have an example for this.
2. If the model is part of your container, then you won't be able to create a new SageMaker model from it. We do have an example notebook showing how to train an XGBoost model in your notebook instance and then host that using SageMaker's XGBoost container. Hopefully this helps clear up how that might be done.
3. In most cases, the model artifacts from SageMaker's built-in algorithms are serialized MXNet objects. In some cases, this means you can load them back into MXNet and do prediction there. However, this is not true for all cases. For example, the model artifact for k-means is an NDArray of cluster centroids. There's no .predict() method associated with this in MXNet. So, in order to do predictions locally, you'd need to write the code that compares an observation to each centroid, finds the closest, and then assigns that centroid to the observation. When using the pre-built MXNet container, you control the model artifact with the save() function in your entry_point script and so you may have more flexibility there to save in a format that easily allows you to generate predictions locally.

[giggle0312]

@djarpin Thank you very much for your full explaination.
1、For the first case, the difference of docker image packaged by myself and ouput image of creating training job on sagemkaer like ****.dkr.ecr.us-east-1.amazonaws.com/linear-learner:latest?
2、I read the XGBOOST example as your advice carefully. However, another question was raised that If the algorithm was open-source depending on scikit-learn packages without corresponding docker containers in sagmaker, for example "decision-tree". Can I run creating new model directly without creating new trainng job. The model artifacts with pkl format can be deserialized by sagemaker?
3、According to your guidance, I wan to explain the third question in my way that sagemaker high-level api based on mxnet for customers using to make prediction ?

[djarpin]

Hi @giggle0312

1. The built-in SageMaker algorithm containers cannot be pulled locally. So, if you trained using SageMaker's Linear Learner algorithm, you could take the model.tar.gz to a hosting solution outside of SageMaker, but you couldn't take the container itself. That means, you'd need to write your own scoring code to generate the predictions. Please see this forum post for more details on how to open up the model.tar.gz for SageMaker's Linear Learner. If you are bringing your own container, like the scikit example, then yes, you could take this container outside of SageMaker, and with the proper setup, use it to generate predictions outside of SageMaker. See the local_test directory for more detail on how to do this.
2. For the XGBoost example, our SageMaker XGBoost algorithm container deserializes the .pkl files. For the scikit example, the predictor.py file includes code to deserialize the model. So, if you have a model.tar.gz in S3 which contains a model artifact in the format your prediction container is expecting, then yes, you can create a model without running a training job in SageMaker.
3. Sorry, not sure I understand this question. The SageMaker Python SDK high-level API for MXNet works with a pre-built MXNet container that's designed to accept idiomatic MXNet code. Even though the model artifacts from SageMaker's built-in algorithms of usually MXNet objects, those containers and algorithms are unrelated to the SageMaker Python SDK's MXNet classes (e.g. sagemaker.mxnet.MXNet).

[giggle0312]

@djarpin Thank you for your kindly explanation.

Then I also confused that how can I use trained tensorflow model or my own algorithm model not bulit-in algorithm model and corresponding image to run on a virtual machine like ec2 instance we have applied with only setting up docker environment?

[djarpin]

Hi @giggle0312 .

If I understand your question properly, you want to use a bring-your-own algorithm container outside of SageMaker, right? If this is the case, then the local_test directory of the scikit-learn example would be the best source. The *.sh scripts will show you how you might emulate SageMaker training, hosting, and prediction from a non-SageMaker EC2 instance.

Alternatively, you could also use the local mode option from the Python SDK.

Is there a specific use case that you have where you need to not use SageMaker resources?
Thanks.

[djarpin]

Closing due to inactivity. Feel free to re-open if necessary. Thanks.

[haysgithub]

Hello, I am hoping I can get some help converting the sagemaker model "model.tar.gz" artifact with deploy.py. I do not know how to use the deploy.py code to run and convert. I need the convertion so I can build my lambda function. Thanks