Salesforce(SFDC)上に登録されているクライアントや売上情報などをembulkで抽出できるか試してみました。まずはembulk側で標準出力できればOK！これが可能ならDigdagでBigQueryに同期できるはず👀
■mikoto2000/embulk-input-salesforce_bulk
https://github.com/mikoto2000/embulk-input-salesforce_bulk
Salesforce Bulk Apiでクエリ結果をTransactionするみたい💪
■Setting
- gem search plugin
1 2 3 4 5 | $ gem search -r embulk-input-salesforce_bulk *** REMOTE GEMS *** embulk-input-salesforce_bulk (0.2.0) |
- embulk-input-salesforce_bulk
- salesforce_bulk
上記の2つをGemfileに追加します
- add Gemfile/run embulk bundle
1 2 | gem 'embulk-input-salesforce_bulk' gem 'salesforce_bulk' |
1 2 3 4 5 6 7 | $ embulk bundle $ grep salesforce Gemfile.lock embulk-input-salesforce_bulk (0.2.0) salesforce_bulk (1.0.3) embulk-input-salesforce_bulk salesforce_bulk |
- add test-salesforce.yml
1 2 3 4 5 6 7 8 9 10 11 12 13 | in: type: salesforce_bulk userName: xxxxxx password: xxxxxx authEndpointUrl: https://xxxxxx.salesforce.com/services/Soap/u/39.0 objectType: Account pollingIntervalMillisecond: 5000 querySelectFrom: SELECT Name FROM Account limit 10 columns: - {type: string, name: Name} out: type: stdout |
- run embulk
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | k$ embulk run -b /xxxx/digdag/vendor test-salesforce.yml 2019-01-24 19:38:40.438 +0900: Embulk v0.8.35 2019-01-24 19:38:49.855 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-salesforce_bulk (0.2.0) 2019-01-24 19:38:49.896 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=4 / output tasks 2 = input tasks 1 * 2 2019-01-24 19:38:49.905 +0900 [INFO] (0001:transaction): {done: 0 / 1, running: 0} 2019-01-24 19:38:50.149 +0900 [INFO] (0014:task-0000): Try login to 'https://xxxxxxxx.salesforce.com/services/Soap/u/39.0'. 2019-01-24 19:38:51.115 +0900 [INFO] (0014:task-0000): Login success. 2019-01-24 19:38:51.115 +0900 [INFO] (0014:task-0000): Send request : 'SELECT Name FROM Account limit 10' [Check] testx,xxxx, [Check] testx,xxxx, test0000, test0001, test0002, test0003, test0004, test0005, test0006, ~省略~ 2019-01-24 19:38:57.470 +0900 [INFO] (0001:transaction): {done: 1 / 1, running: 0} 2019-01-24 19:38:57.477 +0900 [INFO] (main): Committed. 2019-01-24 19:38:57.477 +0900 [INFO] (main): Next config diff: {"in":{},"out":{}} |
🤟
■まとめ
次はBigQueryに同期！！
タイミングよく試した方がいらっしゃるようです。https://t.co/dCJpx5NzZE ご参考まで
— Hiroyuki Sato (@hiroysato) January 28, 2019
じゃぁそのうちこちらも必要になるかもしれませんね👍。https://t.co/llN8tGNTO3
— Hiroyuki Sato (@hiroysato) January 28, 2019
