用的完整指南)
aclnnBidirectionLSTM雙向LSTM算子從原理到調(diào)用的完整指南【免費下載鏈接】ops-nn本項目是CANN提供的神經(jīng)網(wǎng)絡(luò)類計算算子庫實現(xiàn)網(wǎng)絡(luò)在NPU上加速計算。項目地址: https://gitcode.com/cann/ops-nnaclnnBidirectionLSTM是 CANN 算子庫中面向 Ascend NPU 的循環(huán)神經(jīng)網(wǎng)絡(luò)算子位于 ops-nn 的 rnn 分類下。它接收輸入序列與初始隱藏/cell 狀態(tài)完成雙向長短時記憶LSTM的前向推演返回各時刻隱藏狀態(tài)與序列末端的 cell 狀態(tài)常用于文本等序列數(shù)據(jù)的建模與推理。計算原理雙向LSTM算子調(diào)用的門控機(jī)制LSTM 屬于循環(huán)神經(jīng)網(wǎng)絡(luò)家族中的一種變體通過門來調(diào)控信息在時間步之間的流動。該算子負(fù)責(zé)把整段序列按時間步逐步展開每個時間步都用四組門控參數(shù)對上一時刻狀態(tài)和當(dāng)前輸入做加權(quán)組合再疊加偏置后過激活函數(shù)從而更新隱藏狀態(tài)與 cell 狀態(tài)開啟雙向bidirection時會同時做正向與逆向兩條推演把兩路輸出在特征維上拼接。單個時間步 $t$ 的遞推關(guān)系如下$$ f_t \mathrm{sigm}(W_f [h_{t-1}, x_t] b_f) \ i_t \mathrm{sigm}(W_i [h_{t-1}, x_t] b_i) \ o_t \mathrm{sigm}(W_o [h_{t-1}, x_t] b_o) \ \tilde{c}t \tanh(W_c [h_{t-1}, x_t] b_c) \ c_t f_t \odot c{t-1} i_t \odot \tilde{c}t \ c{o}^{t} \tanh(c_t) \ h_t o_t \odot c_{o}^{t} $$各符號含義逐條說明$x_t \in R^drlblxrh1$對應(yīng)公式里 $x_t$ 項LSTM 單元的輸入向量即當(dāng)前時間步喂進(jìn)來的特征。$f_t \in (0, 1)^{h}$對應(yīng)公式里 $f_t$ 項遺忘門激活向量決定保留多少上一時刻 cell 狀態(tài)。$i_t \in (0, 1)^{h}$對應(yīng)公式里 $i_t$ 項輸入門更新門激活向量決定寫入多少新信息。$o_t \in (0, 1)^{h}$對應(yīng)公式里 $o_t$ 項輸出門激活向量控制 cell 狀態(tài)向隱藏狀態(tài)的放行比例。$h_i \in (-1, 1)^{h}$隱藏狀態(tài)向量也就是該時刻 LSTM 單元對外輸出的向量。$\tilde{c}_t \in (-1, 1)^{h}$cell 輸入激活向量經(jīng) tanh 約束后的候選新狀態(tài)。$c_t \in R^{h}$cell 狀態(tài)向量沿時間步累積的長期記憶。$W \in R^{h\times d}$$(U \in R^{h\times h}) \cap (b \in R^{h})$訓(xùn)練階段需要學(xué)習(xí)的權(quán)重矩陣與偏置向量參數(shù)。硬件適配一覽該算子目前僅覆蓋 Atlas 推理系列產(chǎn)品其余主流訓(xùn)練/推理系列均不在支持范圍內(nèi)。支持狀態(tài)以 ? 表示可用、以 × 表示不可用下表將可用項前置便于快速核對。產(chǎn)品系列支持狀態(tài)補(bǔ)充說明Atlas 推理系列產(chǎn)品?當(dāng)前唯一可用產(chǎn)品線Ascend 950PR / Ascend 950DT×不支持Atlas A3 訓(xùn)練/推理系列產(chǎn)品×不支持Atlas A2 訓(xùn)練/推理系列產(chǎn)品×不支持Atlas 200I / 500 A2 推理產(chǎn)品×不支持Atlas 訓(xùn)練系列產(chǎn)品×不支持調(diào)用時序與接口定義aclnn 兩段式接口aclnnBidirectionLSTM遵循 CANN 算子的 aclnn 兩段式接口規(guī)范先由aclnnBidirectionLSTMGetWorkspaceSize完成入?yún)⑿ｒ灢⑺愠鏊?CANN 算子 workspace 的大小、返回執(zhí)行器再由aclnnBidirectionLSTM攜帶 workspace 與執(zhí)行器真正下發(fā)計算。兩段之間不允許插入其他算子調(diào)用。文字時序可概括為應(yīng)用側(cè) CANN 算子 |--- aclnnBidirectionLSTMGetWorkspaceSize(入?yún)? --| 校驗 shape / 類型 / 屬性 |-- workspaceSize executor ----------------------| | 按 workspaceSize 在 device 側(cè)申請內(nèi)存 | |--- aclnnBidirectionLSTM(workspace, executor, stream) --| 執(zhí)行雙向 LSTM 前向 |-- aclnnStatus ------------------------------------|aclnnBidirectionLSTMGetWorkspaceSize函數(shù)原型aclnnStatus aclnnBidirectionLSTMGetWorkspaceSize( const aclTensor *x, const aclTensor *initH, const aclTensor *initC, const aclTensor *wIh, const aclTensor *wHh, const aclTensor *bIhOptional, const aclTensor *bHhOptional, const aclTensor *wIhReverseOptional, const aclTensor *wHhReverseOptional, const aclTensor *bIhReverseOptional, const aclTensor *bHhReverseOptional, int64_t numLayers, bool isbias, bool batchFirst, bool bidirection, const aclTensor *yOut, const aclTensor *outputHOut, const aclTensor *outputCOut, uint64_t *workspaceSize, aclOpExecutor **executor)參數(shù)一覽參數(shù)名方向類型Shape / 取值說明x輸入aclTensor*(time_step, batch_size, input_size)3 維對應(yīng)公式中 $x$ 的輸入向量支持空 TensorFLOAT16ND支持非連續(xù)initH輸入aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維初始化 hidden 狀態(tài)即公式中 $h$支持空 TensorFLOAT16ND支持非連續(xù)initC輸入aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維初始化 cell 狀態(tài)即公式中 $c$支持空 TensorFLOAT16ND支持非連續(xù)wIh輸入aclTensor*(4*hidden_size, input_size)2 維input-hidden 權(quán)重即公式中 $W$支持空 TensorFLOAT16ND支持非連續(xù)wHh輸入aclTensor*(4*hidden_size, hidden_size)2 維hidden-hidden 權(quán)重即公式中 $W$支持空 TensorFLOAT16ND支持非連續(xù)bIhOptional輸入aclTensor*(4*hidden_size)1 維input-hidden 偏置即公式中 $b$支持空 TensorFLOAT16ND支持非連續(xù)bHhOptional輸入aclTensor*(4*hidden_size)1 維hidden-hidden 偏置即公式中 $b$支持空 TensorFLOAT16ND支持非連續(xù)wIhReverseOptional輸入aclTensor*(4*hidden_size, input_size)2 維逆向 input-hidden 權(quán)重支持空 TensorFLOAT16ND支持非連續(xù)wHhReverseOptional輸入aclTensor*(4*hidden_size, input_size)2 維逆向 hidden-hidden 權(quán)重支持空 TensorFLOAT16ND支持非連續(xù)bIhReverseOptional輸入aclTensor*(4*hidden_size)1 維逆向 input-hidden 偏置支持空 TensorFLOAT16ND支持非連續(xù)bHhReverseOptional輸入aclTensor*(4*hidden_size)1 維逆向 hidden-hidden 偏置支持空 TensorFLOAT16ND支持非連續(xù)numLayers輸入int64_t當(dāng)前僅支持 1LSTM 層數(shù)isbias輸入booltrue / false是否帶 biasbatchFirst輸入bool當(dāng)前僅支持 falsebatch 是否位于首維bidirection輸入booltrue / false是否雙向yOut輸出aclTensor*(time_step, batch_size, hidden_size)雙向時 (time_step, batch_size, 2*hidden_size)3 維LSTM 單元輸出向量FLOAT16ND支持非連續(xù)outputHOut輸出aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維最終 hidden 狀態(tài)即公式中 $h$FLOAT16ND支持非連續(xù)outputCOut輸出aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維最終 cell 狀態(tài)即公式中 $c$FLOAT16ND支持非連續(xù)workspaceSize輸出uint64_t*-返回需在 device 側(cè)申請的 workspace 大小executor輸出aclOpExecutor**-返回 op 執(zhí)行器封裝算子計算流程返回值為aclnnStatus狀態(tài)碼。第一段接口會完成入?yún)⑿ｒ炗|發(fā)報錯的典型場景見下文錯誤碼速查。aclnnBidirectionLSTM函數(shù)原型aclnnStatus aclnnBidirectionLSTM( void *workspace, uint64_t workspaceSize, aclOpExecutor *executor, aclrtStream stream)參數(shù)一覽參數(shù)名方向類型說明workspace輸入void*在 device 側(cè)申請的 workspace 內(nèi)存地址workspaceSize輸入uint64_t在 device 側(cè)申請的 workspace 大小由第一段接口返回executor輸入aclOpExecutor*op 執(zhí)行器封裝算子計算流程stream輸入aclrtStream指定執(zhí)行任務(wù)的 Stream返回值為aclnnStatus狀態(tài)碼成功時對應(yīng) ACL_SUCCESS。錯誤碼速查第一段接口aclnnBidirectionLSTMGetWorkspaceSize在入?yún)⑿ｒ炿A段可能返回下列狀態(tài)碼均對應(yīng)aclnnStatus。常量名數(shù)字碼觸發(fā)條件ACLNN_ERR_PARAM_NULLPTR161001傳入的必選輸入、必選輸出或必選屬性為空指針ACLNN_ERR_PARAM_INVALID161002傳入的 aclTensor 數(shù)據(jù)類型不在支持范圍本算子僅 FLOAT16ACLNN_ERR_INNER_TILING_ERROR561002傳入的 aclTensor shape 與上文參數(shù)說明不符約束與限制??numLayers當(dāng)前只支持 1層數(shù)無法通過該參數(shù)疊加。??batchFirst當(dāng)前只支持 false即時間步必須位于首維若傳入 true 會觸發(fā) tiling 階段錯誤。全部張量數(shù)據(jù)類型僅支持 FLOAT16數(shù)據(jù)格式為 ND。雙向模式下initH / initC / outputHOut / outputCOut 的首維為2 * num_layersyOut 的特征維為2 * hidden_size正向模式下均為num_layers與hidden_size。確定性aclnnBidirectionLSTM默認(rèn)即為確定性實現(xiàn)無需額外開關(guān)。各輸入張量與 yOut / outputHOut / outputCOut 均允許非連續(xù)布局也允許以空 Tensor 傳入。完整調(diào)用示例CANN 算子 workspace 的申請與執(zhí)行下面示例演示了從初始化 device、構(gòu)造輸入輸出張量到調(diào)用兩段式接口、同步與釋放資源的完整鏈路。代碼僅為調(diào)用流程參考編譯與鏈接細(xì)節(jié)請以官方文檔為準(zhǔn)。#include iostream #include vector #include acl/acl.h #include aclnnop/aclnn_bidirection_lstm.h #define CHECK_RET(cond, return_expr) \ do { \ if (!(cond)) { \ return_expr; \ } \ } while (0) #define LOG_PRINT(message, ...) \ do { \ printf(message, ##__VA_ARGS__); \ } while (0) int64_t GetShapeSize(const std::vectorint64_t shape) { int64_t shapeSize 1; for (auto i : shape) { shapeSize * i; } return shapeSize; } template typename T int CreateAclTensor(const std::vectorT hostData, const std::vectorint64_t shape, void** deviceAddr, aclDataType dataType, aclTensor** tensor) { auto size GetShapeSize(shape) * sizeof(T); auto ret aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtMalloc failed. ERROR: %d\n, ret); return ret); ret aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtMemcpy failed. ERROR: %d\n, ret); return ret); std::vectorint64_t strides(shape.size(), 1); for (int64_t i shape.size() - 2; i 0; i--) { strides[i] shape[i 1] * strides[i 1]; } *tensor aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(), *deviceAddr); return 0; } int Init(int32_t deviceId, aclrtStream* stream) { auto ret aclInit(nullptr); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclInit failed. ERROR: %d\n, ret); return ret); ret aclrtSetDevice(deviceId); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtSetDevice failed. ERROR: %d\n, ret); return ret); ret aclrtCreateStream(stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtCreateStream failed. ERROR: %d\n, ret); return ret); return 0; } int main() { int32_t deviceId 0; aclrtStream stream; auto ret Init(deviceId, stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(Init acl failed. ERROR: %d\n, ret); return ret); int time_step 2, batch_size 32, input_size 32, hidden_size 32; int64_t numLayers 1; bool isbias true, batchFirst false, bidirection true; std::vectorint64_t selfShape {time_step, batch_size, input_size}; std::vectorint64_t weightHIShape {4 * hidden_size, input_size}; std::vectorint64_t weightHHShape {4 * hidden_size, hidden_size}; std::vectorint64_t initHShape {2, batch_size, hidden_size}; std::vectorint64_t initCShape {2, batch_size, hidden_size}; std::vectorint64_t biasHIShape {4 * hidden_size}; std::vectorint64_t biasHHShape {4 * hidden_size}; std::vectorint64_t outShape {time_step, batch_size, 2 * hidden_size}; std::vectorint64_t outHShape {2, batch_size, hidden_size}; std::vectorint64_t outCShape {2, batch_size, hidden_size}; void* selfDeviceAddr nullptr; void* weightHIDeviceAddr nullptr; void* weightHHDeviceAddr nullptr; void* weightHIReverseDeviceAddr nullptr; void* weightHHReverseDeviceAddr nullptr; void* initHDeviceAddr nullptr; void* initCDeviceAddr nullptr; void* biasHIDeviceAddr nullptr; void* biasHHDeviceAddr nullptr; void* biasHIReverseDeviceAddr nullptr; void* biasHHReverseDeviceAddr nullptr; void* outDeviceAddr nullptr; void* outHDeviceAddr nullptr; void* outCDeviceAddr nullptr; aclTensor* self nullptr; aclTensor* weightHI nullptr; aclTensor* weightHH nullptr; aclTensor* weightHIReverse nullptr; aclTensor* weightHHReverse nullptr; aclTensor* biasHI nullptr; aclTensor* biasHH nullptr; aclTensor* biasHIReverse nullptr; aclTensor* biasHHReverse nullptr; aclTensor* initH nullptr; aclTensor* initC nullptr; aclTensor* out nullptr; aclTensor* outH nullptr; aclTensor* outC nullptr; std::vectoruint16_t selfHostData(GetShapeSize(selfShape)); std::vectoruint16_t weightHIHostData(GetShapeSize(weightHIShape)); std::vectoruint16_t weightHHHostData(GetShapeSize(weightHHShape)); std::vectoruint16_t biasHIHostData(GetShapeSize(biasHIShape)); std::vectoruint16_t biasHHHostData(GetShapeSize(biasHHShape)); std::vectoruint16_t initHHostData(GetShapeSize(initHShape)); std::vectoruint16_t initCHostData(GetShapeSize(initCShape)); std::vectoruint16_t outHostData(GetShapeSize(outShape)); std::vectoruint16_t outHHostData(GetShapeSize(outHShape)); std::vectoruint16_t outCHostData(GetShapeSize(outCShape)); ret CreateAclTensor(selfHostData, selfShape, selfDeviceAddr, aclDataType::ACL_FLOAT16, self); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHIHostData, weightHIShape, weightHIDeviceAddr, aclDataType::ACL_FLOAT16, weightHI); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHHHostData, weightHHShape, weightHHDeviceAddr, aclDataType::ACL_FLOAT16, weightHH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(initHHostData, initHShape, initHDeviceAddr, aclDataType::ACL_FLOAT16, initH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(initCHostData, initCShape, initCDeviceAddr, aclDataType::ACL_FLOAT16, initC); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHIHostData, biasHIShape, biasHIDeviceAddr, aclDataType::ACL_FLOAT16, biasHI); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHHHostData, biasHHShape, biasHHDeviceAddr, aclDataType::ACL_FLOAT16, biasHH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHIHostData, weightHIShape, weightHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, weightHIReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHHHostData, weightHHShape, weightHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, weightHHReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHIHostData, biasHIShape, biasHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, biasHIReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHHHostData, biasHHShape, biasHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, biasHHReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(outHostData, outShape, outDeviceAddr, aclDataType::ACL_FLOAT16, out); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(outHHostData, outHShape, outHDeviceAddr, aclDataType::ACL_FLOAT16, outH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(outCHostData, outCShape, outCDeviceAddr, aclDataType::ACL_FLOAT16, outC); CHECK_RET(ret ACL_SUCCESS, return ret); uint64_t workspaceSize 0; aclOpExecutor* executor nullptr; ret aclnnBidirectionLSTMGetWorkspaceSize(self, initH, initC, weightHI, weightHH, biasHI, biasHH, weightHIReverse, weightHHReverse, biasHIReverse, biasHHReverse, numLayers, isbias, batchFirst, bidirection, out, outH, outC, workspaceSize, executor); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclnnBidirectionLSTMGetWorkspaceSize failed. ERROR: %d\n, ret); return ret); void* workspaceAddr nullptr; if (workspaceSize 0) { ret aclrtMalloc(workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(allocate workspace failed. ERROR: %d\n, ret); return ret); } ret aclnnBidirectionLSTM(workspaceAddr, workspaceSize, executor, stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclnnBidirectionLSTM failed. ERROR: %d\n, ret); return ret); ret aclrtSynchronizeStream(stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtSynchronizeStream failed. ERROR: %d\n, ret); return ret); aclDestroyTensor(self); aclDestroyTensor(weightHI); aclDestroyTensor(weightHH); aclDestroyTensor(initH); aclDestroyTensor(initC); aclDestroyTensor(biasHI); aclDestroyTensor(biasHH); aclDestroyTensor(weightHIReverse); aclDestroyTensor(weightHHReverse); aclDestroyTensor(biasHIReverse); aclDestroyTensor(biasHHReverse); aclDestroyTensor(out); aclDestroyTensor(outH); aclDestroyTensor(outC); aclrtFree(selfDeviceAddr); aclrtFree(weightHIDeviceAddr); aclrtFree(weightHHDeviceAddr); aclrtFree(initHDeviceAddr); aclrtFree(initCDeviceAddr); aclrtFree(biasHIDeviceAddr); aclrtFree(biasHHDeviceAddr); aclrtFree(weightHIReverseDeviceAddr); aclrtFree(weightHHReverseDeviceAddr); aclrtFree(biasHIReverseDeviceAddr); aclrtFree(biasHHReverseDeviceAddr); aclrtFree(outDeviceAddr); aclrtFree(outHDeviceAddr); aclrtFree(outCDeviceAddr); if (workspaceSize 0) { aclrtFree(workspaceAddr); } aclrtDestroyStream(stream); aclrtResetDevice(deviceId); aclFinalize(); return 0; }延伸閱讀算子文檔aclnnBidirectionLSTM.md與本文信息保持一致適合交叉核對參數(shù)約束。反向/雙向前向 kernel 實現(xiàn)op_kernel 目錄其中 lstm_bidir_fp16.h 給出了 FP16 雙向前向的底層組織。tiling 與 op 注冊入口bidirection_lstm.h可幫助理解 shape 校驗與 workspace 估算的來源?？删幾g的運行樣例test_aclnn_bidirection_lstm.cpp。【免費下載鏈接】ops-nn本項目是CANN提供的神經(jīng)網(wǎng)絡(luò)類計算算子庫實現(xiàn)網(wǎng)絡(luò)在NPU上加速計算。項目地址: https://gitcode.com/cann/ops-nn創(chuàng)作聲明：本文部分內(nèi)容由AI輔助生成（AIGC），僅供參考 aclnnBidirectionLSTM：雙向LSTM算子從原理到調(diào)用的完整指南
aclnnBidirectionLSTM雙向LSTM算子從原理到調(diào)用的完整指南【免費下載鏈接】ops-nn本項目是CANN提供的神經(jīng)網(wǎng)絡(luò)類計算算子庫實現(xiàn)網(wǎng)絡(luò)在NPU上加速計算。項目地址: https://gitcode.com/cann/ops-nnaclnnBidirectionLSTM是 CANN 算子庫中面向 Ascend NPU 的循環(huán)神經(jīng)網(wǎng)絡(luò)算子位于 ops-nn 的 rnn 分類下。它接收輸入序列與初始隱藏/cell 狀態(tài)完成雙向長短時記憶LSTM的前向推演返回各時刻隱藏狀態(tài)與序列末端的 cell 狀態(tài)常用于文本等序列數(shù)據(jù)的建模與推理。計算原理雙向LSTM算子調(diào)用的門控機(jī)制LSTM 屬于循環(huán)神經(jīng)網(wǎng)絡(luò)家族中的一種變體通過門來調(diào)控信息在時間步之間的流動。該算子負(fù)責(zé)把整段序列按時間步逐步展開每個時間步都用四組門控參數(shù)對上一時刻狀態(tài)和當(dāng)前輸入做加權(quán)組合再疊加偏置后過激活函數(shù)從而更新隱藏狀態(tài)與 cell 狀態(tài)開啟雙向bidirection時會同時做正向與逆向兩條推演把兩路輸出在特征維上拼接。單個時間步 $t$ 的遞推關(guān)系如下$$ f_t \mathrm{sigm}(W_f [h_{t-1}, x_t] b_f) \ i_t \mathrm{sigm}(W_i [h_{t-1}, x_t] b_i) \ o_t \mathrm{sigm}(W_o [h_{t-1}, x_t] b_o) \ \tilde{c}t \tanh(W_c [h_{t-1}, x_t] b_c) \ c_t f_t \odot c{t-1} i_t \odot \tilde{c}t \ c{o}^{t} \tanh(c_t) \ h_t o_t \odot c_{o}^{t} $$各符號含義逐條說明$x_t \in R^drlblxrh1$對應(yīng)公式里 $x_t$ 項LSTM 單元的輸入向量即當(dāng)前時間步喂進(jìn)來的特征。$f_t \in (0, 1)^{h}$對應(yīng)公式里 $f_t$ 項遺忘門激活向量決定保留多少上一時刻 cell 狀態(tài)。$i_t \in (0, 1)^{h}$對應(yīng)公式里 $i_t$ 項輸入門更新門激活向量決定寫入多少新信息。$o_t \in (0, 1)^{h}$對應(yīng)公式里 $o_t$ 項輸出門激活向量控制 cell 狀態(tài)向隱藏狀態(tài)的放行比例。$h_i \in (-1, 1)^{h}$隱藏狀態(tài)向量也就是該時刻 LSTM 單元對外輸出的向量。$\tilde{c}_t \in (-1, 1)^{h}$cell 輸入激活向量經(jīng) tanh 約束后的候選新狀態(tài)。$c_t \in R^{h}$cell 狀態(tài)向量沿時間步累積的長期記憶。$W \in R^{h\times d}$$(U \in R^{h\times h}) \cap (b \in R^{h})$訓(xùn)練階段需要學(xué)習(xí)的權(quán)重矩陣與偏置向量參數(shù)。硬件適配一覽該算子目前僅覆蓋 Atlas 推理系列產(chǎn)品其余主流訓(xùn)練/推理系列均不在支持范圍內(nèi)。支持狀態(tài)以 ? 表示可用、以 × 表示不可用下表將可用項前置便于快速核對。產(chǎn)品系列支持狀態(tài)補(bǔ)充說明Atlas 推理系列產(chǎn)品?當(dāng)前唯一可用產(chǎn)品線Ascend 950PR / Ascend 950DT×不支持Atlas A3 訓(xùn)練/推理系列產(chǎn)品×不支持Atlas A2 訓(xùn)練/推理系列產(chǎn)品×不支持Atlas 200I / 500 A2 推理產(chǎn)品×不支持Atlas 訓(xùn)練系列產(chǎn)品×不支持調(diào)用時序與接口定義aclnn 兩段式接口aclnnBidirectionLSTM遵循 CANN 算子的 aclnn 兩段式接口規(guī)范先由aclnnBidirectionLSTMGetWorkspaceSize完成入?yún)⑿ｒ灢⑺愠鏊?CANN 算子 workspace 的大小、返回執(zhí)行器再由aclnnBidirectionLSTM攜帶 workspace 與執(zhí)行器真正下發(fā)計算。兩段之間不允許插入其他算子調(diào)用。文字時序可概括為應(yīng)用側(cè) CANN 算子 |--- aclnnBidirectionLSTMGetWorkspaceSize(入?yún)? --| 校驗 shape / 類型 / 屬性 |-- workspaceSize executor ----------------------| | 按 workspaceSize 在 device 側(cè)申請內(nèi)存 | |--- aclnnBidirectionLSTM(workspace, executor, stream) --| 執(zhí)行雙向 LSTM 前向 |-- aclnnStatus ------------------------------------|aclnnBidirectionLSTMGetWorkspaceSize函數(shù)原型aclnnStatus aclnnBidirectionLSTMGetWorkspaceSize( const aclTensor *x, const aclTensor *initH, const aclTensor *initC, const aclTensor *wIh, const aclTensor *wHh, const aclTensor *bIhOptional, const aclTensor *bHhOptional, const aclTensor *wIhReverseOptional, const aclTensor *wHhReverseOptional, const aclTensor *bIhReverseOptional, const aclTensor *bHhReverseOptional, int64_t numLayers, bool isbias, bool batchFirst, bool bidirection, const aclTensor *yOut, const aclTensor *outputHOut, const aclTensor *outputCOut, uint64_t *workspaceSize, aclOpExecutor **executor)參數(shù)一覽參數(shù)名方向類型Shape / 取值說明x輸入aclTensor*(time_step, batch_size, input_size)3 維對應(yīng)公式中 $x$ 的輸入向量支持空 TensorFLOAT16ND支持非連續(xù)initH輸入aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維初始化 hidden 狀態(tài)即公式中 $h$支持空 TensorFLOAT16ND支持非連續(xù)initC輸入aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維初始化 cell 狀態(tài)即公式中 $c$支持空 TensorFLOAT16ND支持非連續(xù)wIh輸入aclTensor*(4*hidden_size, input_size)2 維input-hidden 權(quán)重即公式中 $W$支持空 TensorFLOAT16ND支持非連續(xù)wHh輸入aclTensor*(4*hidden_size, hidden_size)2 維hidden-hidden 權(quán)重即公式中 $W$支持空 TensorFLOAT16ND支持非連續(xù)bIhOptional輸入aclTensor*(4*hidden_size)1 維input-hidden 偏置即公式中 $b$支持空 TensorFLOAT16ND支持非連續(xù)bHhOptional輸入aclTensor*(4*hidden_size)1 維hidden-hidden 偏置即公式中 $b$支持空 TensorFLOAT16ND支持非連續(xù)wIhReverseOptional輸入aclTensor*(4*hidden_size, input_size)2 維逆向 input-hidden 權(quán)重支持空 TensorFLOAT16ND支持非連續(xù)wHhReverseOptional輸入aclTensor*(4*hidden_size, input_size)2 維逆向 hidden-hidden 權(quán)重支持空 TensorFLOAT16ND支持非連續(xù)bIhReverseOptional輸入aclTensor*(4*hidden_size)1 維逆向 input-hidden 偏置支持空 TensorFLOAT16ND支持非連續(xù)bHhReverseOptional輸入aclTensor*(4*hidden_size)1 維逆向 hidden-hidden 偏置支持空 TensorFLOAT16ND支持非連續(xù)numLayers輸入int64_t當(dāng)前僅支持 1LSTM 層數(shù)isbias輸入booltrue / false是否帶 biasbatchFirst輸入bool當(dāng)前僅支持 falsebatch 是否位于首維bidirection輸入booltrue / false是否雙向yOut輸出aclTensor*(time_step, batch_size, hidden_size)雙向時 (time_step, batch_size, 2*hidden_size)3 維LSTM 單元輸出向量FLOAT16ND支持非連續(xù)outputHOut輸出aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維最終 hidden 狀態(tài)即公式中 $h$FLOAT16ND支持非連續(xù)outputCOut輸出aclTensor*(num_layers, batch_size, hidden_size)雙向時 (2*num_layers, ...)3 維最終 cell 狀態(tài)即公式中 $c$FLOAT16ND支持非連續(xù)workspaceSize輸出uint64_t*-返回需在 device 側(cè)申請的 workspace 大小executor輸出aclOpExecutor**-返回 op 執(zhí)行器封裝算子計算流程返回值為aclnnStatus狀態(tài)碼。第一段接口會完成入?yún)⑿ｒ炗|發(fā)報錯的典型場景見下文錯誤碼速查。aclnnBidirectionLSTM函數(shù)原型aclnnStatus aclnnBidirectionLSTM( void *workspace, uint64_t workspaceSize, aclOpExecutor *executor, aclrtStream stream)參數(shù)一覽參數(shù)名方向類型說明workspace輸入void*在 device 側(cè)申請的 workspace 內(nèi)存地址workspaceSize輸入uint64_t在 device 側(cè)申請的 workspace 大小由第一段接口返回executor輸入aclOpExecutor*op 執(zhí)行器封裝算子計算流程stream輸入aclrtStream指定執(zhí)行任務(wù)的 Stream返回值為aclnnStatus狀態(tài)碼成功時對應(yīng) ACL_SUCCESS。錯誤碼速查第一段接口aclnnBidirectionLSTMGetWorkspaceSize在入?yún)⑿ｒ炿A段可能返回下列狀態(tài)碼均對應(yīng)aclnnStatus。常量名數(shù)字碼觸發(fā)條件ACLNN_ERR_PARAM_NULLPTR161001傳入的必選輸入、必選輸出或必選屬性為空指針ACLNN_ERR_PARAM_INVALID161002傳入的 aclTensor 數(shù)據(jù)類型不在支持范圍本算子僅 FLOAT16ACLNN_ERR_INNER_TILING_ERROR561002傳入的 aclTensor shape 與上文參數(shù)說明不符約束與限制??numLayers當(dāng)前只支持 1層數(shù)無法通過該參數(shù)疊加。??batchFirst當(dāng)前只支持 false即時間步必須位于首維若傳入 true 會觸發(fā) tiling 階段錯誤。全部張量數(shù)據(jù)類型僅支持 FLOAT16數(shù)據(jù)格式為 ND。雙向模式下initH / initC / outputHOut / outputCOut 的首維為2 * num_layersyOut 的特征維為2 * hidden_size正向模式下均為num_layers與hidden_size。確定性aclnnBidirectionLSTM默認(rèn)即為確定性實現(xiàn)無需額外開關(guān)。各輸入張量與 yOut / outputHOut / outputCOut 均允許非連續(xù)布局也允許以空 Tensor 傳入。完整調(diào)用示例CANN 算子 workspace 的申請與執(zhí)行下面示例演示了從初始化 device、構(gòu)造輸入輸出張量到調(diào)用兩段式接口、同步與釋放資源的完整鏈路。代碼僅為調(diào)用流程參考編譯與鏈接細(xì)節(jié)請以官方文檔為準(zhǔn)。#include iostream #include vector #include acl/acl.h #include aclnnop/aclnn_bidirection_lstm.h #define CHECK_RET(cond, return_expr) \ do { \ if (!(cond)) { \ return_expr; \ } \ } while (0) #define LOG_PRINT(message, ...) \ do { \ printf(message, ##__VA_ARGS__); \ } while (0) int64_t GetShapeSize(const std::vectorint64_t shape) { int64_t shapeSize 1; for (auto i : shape) { shapeSize * i; } return shapeSize; } template typename T int CreateAclTensor(const std::vectorT hostData, const std::vectorint64_t shape, void** deviceAddr, aclDataType dataType, aclTensor** tensor) { auto size GetShapeSize(shape) * sizeof(T); auto ret aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtMalloc failed. ERROR: %d\n, ret); return ret); ret aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtMemcpy failed. ERROR: %d\n, ret); return ret); std::vectorint64_t strides(shape.size(), 1); for (int64_t i shape.size() - 2; i 0; i--) { strides[i] shape[i 1] * strides[i 1]; } *tensor aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(), *deviceAddr); return 0; } int Init(int32_t deviceId, aclrtStream* stream) { auto ret aclInit(nullptr); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclInit failed. ERROR: %d\n, ret); return ret); ret aclrtSetDevice(deviceId); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtSetDevice failed. ERROR: %d\n, ret); return ret); ret aclrtCreateStream(stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtCreateStream failed. ERROR: %d\n, ret); return ret); return 0; } int main() { int32_t deviceId 0; aclrtStream stream; auto ret Init(deviceId, stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(Init acl failed. ERROR: %d\n, ret); return ret); int time_step 2, batch_size 32, input_size 32, hidden_size 32; int64_t numLayers 1; bool isbias true, batchFirst false, bidirection true; std::vectorint64_t selfShape {time_step, batch_size, input_size}; std::vectorint64_t weightHIShape {4 * hidden_size, input_size}; std::vectorint64_t weightHHShape {4 * hidden_size, hidden_size}; std::vectorint64_t initHShape {2, batch_size, hidden_size}; std::vectorint64_t initCShape {2, batch_size, hidden_size}; std::vectorint64_t biasHIShape {4 * hidden_size}; std::vectorint64_t biasHHShape {4 * hidden_size}; std::vectorint64_t outShape {time_step, batch_size, 2 * hidden_size}; std::vectorint64_t outHShape {2, batch_size, hidden_size}; std::vectorint64_t outCShape {2, batch_size, hidden_size}; void* selfDeviceAddr nullptr; void* weightHIDeviceAddr nullptr; void* weightHHDeviceAddr nullptr; void* weightHIReverseDeviceAddr nullptr; void* weightHHReverseDeviceAddr nullptr; void* initHDeviceAddr nullptr; void* initCDeviceAddr nullptr; void* biasHIDeviceAddr nullptr; void* biasHHDeviceAddr nullptr; void* biasHIReverseDeviceAddr nullptr; void* biasHHReverseDeviceAddr nullptr; void* outDeviceAddr nullptr; void* outHDeviceAddr nullptr; void* outCDeviceAddr nullptr; aclTensor* self nullptr; aclTensor* weightHI nullptr; aclTensor* weightHH nullptr; aclTensor* weightHIReverse nullptr; aclTensor* weightHHReverse nullptr; aclTensor* biasHI nullptr; aclTensor* biasHH nullptr; aclTensor* biasHIReverse nullptr; aclTensor* biasHHReverse nullptr; aclTensor* initH nullptr; aclTensor* initC nullptr; aclTensor* out nullptr; aclTensor* outH nullptr; aclTensor* outC nullptr; std::vectoruint16_t selfHostData(GetShapeSize(selfShape)); std::vectoruint16_t weightHIHostData(GetShapeSize(weightHIShape)); std::vectoruint16_t weightHHHostData(GetShapeSize(weightHHShape)); std::vectoruint16_t biasHIHostData(GetShapeSize(biasHIShape)); std::vectoruint16_t biasHHHostData(GetShapeSize(biasHHShape)); std::vectoruint16_t initHHostData(GetShapeSize(initHShape)); std::vectoruint16_t initCHostData(GetShapeSize(initCShape)); std::vectoruint16_t outHostData(GetShapeSize(outShape)); std::vectoruint16_t outHHostData(GetShapeSize(outHShape)); std::vectoruint16_t outCHostData(GetShapeSize(outCShape)); ret CreateAclTensor(selfHostData, selfShape, selfDeviceAddr, aclDataType::ACL_FLOAT16, self); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHIHostData, weightHIShape, weightHIDeviceAddr, aclDataType::ACL_FLOAT16, weightHI); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHHHostData, weightHHShape, weightHHDeviceAddr, aclDataType::ACL_FLOAT16, weightHH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(initHHostData, initHShape, initHDeviceAddr, aclDataType::ACL_FLOAT16, initH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(initCHostData, initCShape, initCDeviceAddr, aclDataType::ACL_FLOAT16, initC); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHIHostData, biasHIShape, biasHIDeviceAddr, aclDataType::ACL_FLOAT16, biasHI); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHHHostData, biasHHShape, biasHHDeviceAddr, aclDataType::ACL_FLOAT16, biasHH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHIHostData, weightHIShape, weightHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, weightHIReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(weightHHHostData, weightHHShape, weightHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, weightHHReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHIHostData, biasHIShape, biasHIReverseDeviceAddr, aclDataType::ACL_FLOAT16, biasHIReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(biasHHHostData, biasHHShape, biasHHReverseDeviceAddr, aclDataType::ACL_FLOAT16, biasHHReverse); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(outHostData, outShape, outDeviceAddr, aclDataType::ACL_FLOAT16, out); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(outHHostData, outHShape, outHDeviceAddr, aclDataType::ACL_FLOAT16, outH); CHECK_RET(ret ACL_SUCCESS, return ret); ret CreateAclTensor(outCHostData, outCShape, outCDeviceAddr, aclDataType::ACL_FLOAT16, outC); CHECK_RET(ret ACL_SUCCESS, return ret); uint64_t workspaceSize 0; aclOpExecutor* executor nullptr; ret aclnnBidirectionLSTMGetWorkspaceSize(self, initH, initC, weightHI, weightHH, biasHI, biasHH, weightHIReverse, weightHHReverse, biasHIReverse, biasHHReverse, numLayers, isbias, batchFirst, bidirection, out, outH, outC, workspaceSize, executor); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclnnBidirectionLSTMGetWorkspaceSize failed. ERROR: %d\n, ret); return ret); void* workspaceAddr nullptr; if (workspaceSize 0) { ret aclrtMalloc(workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(allocate workspace failed. ERROR: %d\n, ret); return ret); } ret aclnnBidirectionLSTM(workspaceAddr, workspaceSize, executor, stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclnnBidirectionLSTM failed. ERROR: %d\n, ret); return ret); ret aclrtSynchronizeStream(stream); CHECK_RET(ret ACL_SUCCESS, LOG_PRINT(aclrtSynchronizeStream failed. ERROR: %d\n, ret); return ret); aclDestroyTensor(self); aclDestroyTensor(weightHI); aclDestroyTensor(weightHH); aclDestroyTensor(initH); aclDestroyTensor(initC); aclDestroyTensor(biasHI); aclDestroyTensor(biasHH); aclDestroyTensor(weightHIReverse); aclDestroyTensor(weightHHReverse); aclDestroyTensor(biasHIReverse); aclDestroyTensor(biasHHReverse); aclDestroyTensor(out); aclDestroyTensor(outH); aclDestroyTensor(outC); aclrtFree(selfDeviceAddr); aclrtFree(weightHIDeviceAddr); aclrtFree(weightHHDeviceAddr); aclrtFree(initHDeviceAddr); aclrtFree(initCDeviceAddr); aclrtFree(biasHIDeviceAddr); aclrtFree(biasHHDeviceAddr); aclrtFree(weightHIReverseDeviceAddr); aclrtFree(weightHHReverseDeviceAddr); aclrtFree(biasHIReverseDeviceAddr); aclrtFree(biasHHReverseDeviceAddr); aclrtFree(outDeviceAddr); aclrtFree(outHDeviceAddr); aclrtFree(outCDeviceAddr); if (workspaceSize 0) { aclrtFree(workspaceAddr); } aclrtDestroyStream(stream); aclrtResetDevice(deviceId); aclFinalize(); return 0; }延伸閱讀算子文檔aclnnBidirectionLSTM.md與本文信息保持一致適合交叉核對參數(shù)約束。反向/雙向前向 kernel 實現(xiàn)op_kernel 目錄其中 lstm_bidir_fp16.h 給出了 FP16 雙向前向的底層組織。tiling 與 op 注冊入口bidirection_lstm.h可幫助理解 shape 校驗與 workspace 估算的來源?？删幾g的運行樣例test_aclnn_bidirection_lstm.cpp。【免費下載鏈接】ops-nn本項目是CANN提供的神經(jīng)網(wǎng)絡(luò)類計算算子庫實現(xiàn)網(wǎng)絡(luò)在NPU上加速計算。項目地址: https://gitcode.com/cann/ops-nn創(chuàng)作聲明：本文部分內(nèi)容由AI輔助生成（AIGC），僅供參考