[Erlang 0021] From String To Erlang Code

本文涉及的产品
公共DNS(含HTTPDNS解析),每月1000万次HTTP解析
全局流量管理 GTM,标准版 1个月
云解析 DNS,旗舰版 1个月
简介:

团队的兄弟遇到一个问题,如何把字符串动态解析成为Erlang数据结构(Erlang term)? 比如"[{1,2},{2,3},{3,4}]"转成[{1,2},{2,3},{3,4}]

From String To Erlang Term

http://blog.yufeng.info/archives/tag/erl_eval 余锋在<erlang动态解释>这篇文章中给出了解决方案,他分析的是init.erl模块;按照文章的思路我们在shell里面试一下:

Eshell V5.8.2 (abort with ^G)
1> {ok, Scan1, _} = erl_scan:string("[a,b,c].").
{ok,[{'[',1},{atom,1,a},{',',1},{atom,1,b},{',',1},{atom,1,c},{']',1},{dot,1}],1}
2> {ok,P}=erl_parse:parse_exprs(Scan1).
{ok,[{cons,1,{atom,1,a},{cons,1,{atom,1,b},{cons,1,{atom,1,c},{nil,1}}}}]}
3> erl_eval:exprs(P, []) .
{value,[a,b,c],[]}
4>

红色标注的部分就是我们想要的结果,注意erl_scan:string(Exp).接受的参数是一个合法的表达式,必须以.结尾,代表一个表达式的结束,否则文法检查过不去;看一下输出的结果里面.符号被解析为{dot,1};这个问题最关键的部分就已经解决了,还有一个相关的问题就是如何把一个[{1,2},{2,3},{3,4}]转成字符串?这个当然要在io_lib里面去寻找答案,可以这样做: lists:flatten(io_lib:write([{1,2},{2,3},{3,4}])). 结果为"[{1,2},{2,3},{3,4}]"我们做一个完整的例子:

1> lists:flatten(io_lib:write([{1,2},{2,3},{3,4}])).
"[{1,2},{2,3},{3,4}]"
2> S= lists:flatten(io_lib:write([{1,2},{2,3},{3,4}])).
"[{1,2},{2,3},{3,4}]"
3> E=S++".". %%添加结束符
"[{1,2},{2,3},{3,4}]."
4> {ok, Scan1, _} = erl_scan:string(E).
{ok,[{'[',1},{'{',1},{integer,1,1},{',',1},{integer,1,2},{'}',1},{',',1},{'{',1},{integer,1,2},{',',1},{integer,1,3},{'}',1},{',',1},{'{',1},{integer,1,3},{',',1},{integer,1,4},{'}',1},{']',1},{dot,1}], 1}

5> {ok,P}=erl_parse:parse_exprs(Scan1).
{ok,[{cons,1,{tuple,1,[{integer,1,1},{integer,1,2}]}, {cons,1,{tuple,1,[{integer,1,2},{integer,1,3}]},
{cons,1,{tuple,1,[{integer,1,3},{integer,1,4}]},{nil,1}}}}]}
6> erl_eval:exprs(P, []) .
{value,[{1,2},{2,3},{3,4}],[]}
7>

仅仅是解析Erlang Term 也可以这样:

复制代码
list_to_term(String) ->
{ok, T, _} = erl_scan:string(String++"."),
case erl_parse:parse_term(T) of
{ok, Term} ->
Term;
{error, Error} ->
Error
end.
复制代码

From String To Erlang Code

这个问题可以泛化为字符串解析为Erlang代码并执行?同样的问题在.net中,我们可以这样解决:

复制代码
using System;
using IronPython.Hosting; //缺少引用的去下载一个: http://ironpython.codeplex.com/
using Microsoft.Scripting.Hosting;
namespace ConsoleApplication2
{
class Program
{
static void Main(string[] args)
{
ScriptEngine engine = Python.CreateEngine();
var Result = engine.Execute("2*(1+3)");
Console.WriteLine(Result);
Console.ReadLine();
}
}
}
复制代码

 

还是使用上面的erl_scan erl_parse erl_eval基础设施,我们可以很容易实现:

Eshell V5.8.2 (abort with ^G)
1> E=fun(S) ->
{ok,Scanned,_} = erl_scan:string(S),
{ok,Parsed} = erl_parse:parse_exprs(Scanned),
erl_eval:exprs(Parsed,[]) end.
#Fun<erl_eval.6.13229925>
2> E("R=1+2*3.").
{value,7,[{'R',7}]}
3> E("A=2,B=3,A+B.").

{value,5,[{'A',2},{'B',3}]}

那如果是"M=K+L."这样的表达式呢?如何呢?

4> E("M=K+L.").
** exception error: {unbound_var,'K'}

我们直接在shell里面执行报的错误一致,只不过错误信息格式变化一下:* 1: variable 'K' is unbound,那么如何动态给变量绑定数值呢?这样我们可以更灵活控制表达式和值, http://www.trapexit.org/String_Eval 给了一个这样的例子:

 

-module(test).

-export([test/0]).

test()->
    %% Create a code string with unbound variables 'A' and 'B'
    String="Results=A+B/2.",
    
    %% Scan the code into tokens
    {ok,ErlTokens,_}=erl_scan:string(String),
    io:format("ErlTokens are ~p~n",[ErlTokens]),

    %% Now parse the tokens into the abstract form
    {ok,ErlAbsForm}=erl_parse:parse_exprs(ErlTokens),
    io:format("ErlAbsForm are ~p~n",[ErlAbsForm]),

    %% Now we need to bind values to variable 'A' and 'B'
    Bindings=erl_eval:add_binding('A',20,erl_eval:new_bindings()),
    NewBindings=erl_eval:add_binding('B',45,Bindings),
    io:format("The bindings are ~p~n",[erl_eval:bindings(NewBindings)]),

    %% Now evaluate the string
    io:format("Going into erl_eval:exprs~n",[]),
    {value,Value,_}=erl_eval:exprs(ErlAbsForm,NewBindings),
    io:format("Value is ~p~n",[Value]).

You can compile and run this in the shell:

(arrian@psyduck)17> c(test).
{ok,test}
(arrian@psyduck)18> test:test().
ErlTokens are [{var,1,'Results'},
               {'=',1},
               {var,1,'A'},
               {'+',1},
               {var,1,'B'},
               {'/',1},
               {integer,1,2},
               {dot,1}]
ErlAbsForm are [{match,1,
                       {var,1,'Results'},
                       {op,1,
                           '+',
                           {var,1,'A'},
                           {op,1,'/',{var,1,'B'},{integer,1,2}}}}]
The bindings are [{'A',20},{'B',45}]
Going into erl_eval:exprs
Value is 42.5000
ok
(arrian@psyduck)19>            

Note: If you bind variables that don't exist in the code string/token set/abstract form then when you erl_eval the abstract form will simply silently ignore your additional bindings

 

我曾经介绍过开源项目smerl,其定位就是Simple Metaprogramming for Erlang, 我们可以从这份代码里面学到erl_scan erl_parse erl_eval更灵活的应用,项目地址:http://code.google.com/p/smerl/ 

  test_smerl() ->
      M1 = smerl:new(foo),
      {ok, M2} = smerl:add_func(M1, "bar() -> 1 + 1."),
      smerl:compile(M2),
      foo:bar(),   % returns 2``
      smerl:has_func(M2, bar, 0). % returns true

 最后顺便提一句,如何把{abc}作为字符串写入到文件中呢?  lists:flatten(io_lib:format("~p",[{a,b,c}])).

 

2012年9月27日16:54:11 更新 添加一段类似的代码

复制代码
%%%
%%% distel_ie - an interactive erlang shell
%%%
%%% Some of the code has shamelessly been stolen from Luke Gorrie 
%%% [luke@bluetail.com] - ripped from its elegance and replaced by bugs. 
%%% It just goes to show that you can't trust anyone these days. And
%%% as if that wasn't enough, I'll even blame Luke: "He _made_ me do it!"
%%%
%%% So, without any remorse, I hereby declare this code to be:
%%%
%%% copyright (c) 2002 david wallin [david.wallin@ul.ie].
%%%
%%% (it's probably going to be released onto an unexpecting public under
%%%  some sort of BSD license).

-module(distel_ie).

-export([
     evaluate/2,

     test1/0,
     test2/0,
     test3/0,
     test4/0,
     test5/0,

     ensure_registered/0,

     start/0,
     start/1,
     init/1,
     loop/1
    ]).
-compile(export_all).
-define(FMT(X), list_to_binary(lists:flatten(io_lib:format("~p", [X])))).


%%
%% ensure_registered/0
ensure_registered() ->
    case whereis(distel_ie) of
           undefined -> start() ;
           Pid  ->
              group_leader(group_leader(), Pid),
              ok
    end.

%%
%% start/0

start() ->
    start([]).


%%
%% start/1

start(Options) ->
    spawn(?MODULE, init, [Options]).


%%
%% init/1

init(Options) ->
    register(distel_ie, self()),
    Defs = ets:new(definitions, [set]),
    Line = 12,
    Bindings = [],
    State = {Defs, Line, Bindings},
    loop(State).


%%
%% loop/1

loop({Defs, Line, Bindings}) ->
    receive 
    {evaluate, Emacs, String} -> 
        case catch evaluate(String, {Defs, Line, Bindings}) of
        {'EXIT', Rsn} ->
            Emacs ! {ok, list_to_binary(
                   io_lib:format("EXIT: ~p", [Rsn]))},
            ?MODULE:loop({Defs, Line, Bindings});
        {Result, {NL, NB}} ->
            Emacs ! Result,
            ?MODULE:loop({Defs, NL, NB})
        end;
    forget_bindings ->
        put(distel_ie_bindings, []),
        ?MODULE:loop({Defs, Line, []}) ;
    Unknown ->
        io:format("distel_ie: unknown message recvd '~p'\n", [Unknown]),
        ?MODULE:loop({Defs, Line, Bindings})

    end.


%%
%% evaluate/2

evaluate(String, {Defs, Line, Bindings}) ->
    case parse_expr(String) of
    %% ok, so it is an expression :
    {ok, Parse} ->
        RemoteParse = add_remote_call_info(Parse, Defs),
            case catch erl_eval:exprs(RemoteParse, Bindings) of
                {value, V, NewBinds} ->
            {{ok, ?FMT(V)}, {Line, NewBinds}};
                Error ->
            {?FMT(Error), {Line, Bindings}}
        end;
    %% try and treat it as a form / definition instead :
    Other ->
        case parse_form(String) of
        {ok, Parse} -> 
            {ok, Name, Arity} = get_function_name(Parse),
            ets:insert(Defs, {Name, Parse}),
            FunTrees = lists:flatten(
                 lists:reverse(ets:match(Defs,{'_', '$1'}))),
            %% Line isn't really used yet
            NewLine = Line,
            compile_load(FunTrees),
            Def = list_to_binary(atom_to_list(Name) ++ "/" ++ 
                     integer_to_list(Arity)),
            {{ok, Def}, {NewLine, Bindings}} ;
        Error ->
            {{error, ?FMT({Error, Other})}, {Line, Bindings}}
        end
    end.

%%
%% parse_expr/1

parse_expr(String) ->
    case erl_scan:string(String) of
    {ok, Tokens, _} ->
        catch erl_parse:parse_exprs(Tokens) ;
    {error, {_Line, erl_parse, Rsn}} ->
        {error, lists:flatten(Rsn)};
    {error, Error, _} ->
        {error, Error}
    end.

%%
%% parse_form/1

parse_form(String) ->
    case erl_scan:string(String) of
    {ok, Tokens, _} ->
        catch erl_parse:parse_form(Tokens) ;
    {error, {_Line, erl_parse, Rsn}} ->
        {error, lists:flatten(Rsn)};
    {error, Error, _} ->
        {error, Error}
    end.


%%
%% defun/1

defun(String) ->
    {ok, Tokens, _} = erl_scan:string(String),
    {ok, Parse} = erl_parse:parse_form(Tokens),
    compile_load([Parse]).


%%
%% compile_load/1

compile_load(Parse) ->

    Header = [{attribute,9,module,distel_ie_internal},
          {attribute,11,compile,export_all},
          {attribute,12,export,[]}],

    EOF = [{eof,20}],

    SyntaxTree = Header ++ Parse ++ EOF,

    {ok, Mod, Binary} = compile:forms(SyntaxTree),
    File = "heltigenomfelfelsomfansomentyskindianenbefjadradgerman",
    code:load_binary(Mod, File, Binary).


%%
%% add_remote_call_info/2
%%
%% TODO: this is gonna need more work, e.g. it needs to recurse into 
%% lists (cons) and tuples ... +more

add_remote_call_info([], _Defs) -> [] ;
add_remote_call_info({var, L, Var}, Defs) ->
    {var, L, Var} ;
add_remote_call_info({atom, L, Atom}, Defs) ->
    {atom, L, Atom} ;
add_remote_call_info({integer, L, Value}, Defs) ->
    {integer, L, Value} ;
add_remote_call_info({string, L, String}, Defs) ->
    {string, L, String} ;
add_remote_call_info([{call, L, {atom, L2, Name}, Body} | Rs], Defs) ->
    B = add_remote_call_info(Body, Defs),
    IsBuiltin = erlang:is_builtin(erlang, Name, length(B)),
    Call = case IsBuiltin of 
           true ->
           {call, L, {atom, L2, Name}, B} ;
           false ->
           Arity = length(Body),
           case find_module(Name, Arity) of 
               {ok, Mod} ->
                {call, L, {remote, L2, 
                      {atom, L2, Mod}, 
                      {atom, L2, Name}}, B} ;
               {error, _} ->
               {call, L, {atom, L2, Name}, B}
           end
       end,
    [Call | add_remote_call_info(Rs, Defs)] ;

add_remote_call_info([{tuple, L, Values} | Rs], Defs) ->
    F = fun(X) -> add_remote_call_info(X, Defs) end,
    [{tuple, L, lists:map(F, Values)} | add_remote_call_info(Rs, Defs)] ;


add_remote_call_info([{Type, L, Hdr, Body} | Rs], Defs) when list(Body) ->
    B = add_remote_call_info(Body, Defs),
    [{Type, L, Hdr, B} | add_remote_call_info(Rs, Defs)] ;

add_remote_call_info([{Type, L, Hd, Tl} | Rs], Defs) ->
    [{Type, L, Hd, Tl} | add_remote_call_info(Rs, Defs)] ;

add_remote_call_info([R | Rs], Defs) ->
    [add_remote_call_info(R, Defs) | add_remote_call_info(Rs, Defs) ];

add_remote_call_info(X, Defs) ->
    X.


%%
%% find_module/2

find_module(Function, Arity) ->
    Mods = [distel_ie_internal, distel_ie, c],
    F = fun(M) -> not is_exported(Function, Arity, M) end,
    case lists:dropwhile(F, Mods) of
    [] ->
        search_modules(Function, Arity, code:all_loaded()) ;
    [M | _] ->
        {ok, M}
    end.


%%
%% is_exported/3

is_exported(Function, Arity, Module) ->
    case code:is_loaded(Module) of
    false ->
        false;
    _ ->
        Info = Module:module_info(),
        {value, {exports, Exports}} = lists:keysearch(exports, 1, Info),
        lists:member({Function, Arity}, Exports)
    end.


%%
%% search_modules/3

search_modules(Function, Arity, []) ->
    {error, not_found};
search_modules(Function, Arity, [{M, _} | Ms]) ->
    case is_exported(Function, Arity, M) of
    true ->
        {ok, M} ;
    false ->
        search_modules(Function, Arity, Ms)
    end.
    

%%
%% get_function_name/1

get_function_name({function, _, Name, Arity, _}) -> 
    {ok, Name, Arity} ;

get_function_name(Unknown) ->
    {error, Unknown}.


%%% ------------------------------------------------------------------- [tests]
    
%%
%% test1/0

test1() ->
    Defun = "sista(W) -> lists:last(W).",
    defun(Defun).

%%
%% test2/0

test2() ->
    
    Prefix = [
          {attribute,9,module,compile_and_load_me},
          {attribute,11,compile,export_all},
          {attribute,12,export,[]}
         ],
    
    Postfix = [{eof,20}],
    
    String = "sista([]) -> [] ;\n\nsista(W) -> lists:last(W).\n",
%    String = "sista([], _) -> [] ;\n\nsista(W, _) -> lists:last(W).\n",
    
    {ok, Tokens, _} = erl_scan:string(String, 23),
    {ok, Tree} = erl_parse:parse_form(Tokens),
    
    io:format("tree : '~p'\n", [Tree]),
    
    SyntaxTree = Prefix ++ [Tree] ++ Postfix,
    
    io:format("syntaxtree : '~p'\n", [SyntaxTree]),
    
    {ok, Mod, Binary} = compile:forms(SyntaxTree),
    code:load_binary(Mod, "spam", Binary),
    compile_and_load_me:sista([1,2,galapremiere]).


%%
%% test3/0

test3() ->

    Sista = "sista(W) -> lists:last(W).\n",
    defun(Sista),
    distel_ie_internal:sista([1,2,galapremiere]),
    NySista = "en_till_sista(W) -> lists:last(W).\n",
    defun(NySista),
    distel_ie_internal:en_till_sista([1,2,onestepbeyond]).


%%
%% test4/0

test4() ->

    Defs = ets:new(definitions, [set]),

    Define = "nisse([]) -> [] ;\n\nnisse(W) -> lists:last(W).",
    Expr = "nisse([1,2,3]).",
%    Expr = "distel_ie_internal:nisse([1,2,3]).",
%    Expr = "lists:last([1,2,3]).",
    
    D = evaluate(Define, {Defs, 10, []}),
    io:format("nisse defined: '~p'\n", [D]),
    io:format("is_loaded: '~p'\n", [code:is_loaded(distel_ie_internal)]),
    evaluate(Expr, {Defs, 10, []}).


%%
%% test5/0

test5() ->

    Defs = ets:new(definitions, [set]),
    
    Define = "nisse([]) -> [] ;\n\nnisse(W) -> lists:last(W).",
    Expr = "nisse([1,2,3]).",

    evaluate(Define, {Defs, 10, []}),
%    ets:insert(Defs, {nisse, nil}),

    {ok, Tokens, _} = erl_scan:string(Expr, 23),
    {ok, Tree} = erl_parse:parse_exprs(Tokens),

    io:format("original tree : '~p'\n", [Tree]),
    
    RemoteTree = add_remote_call_info(Tree, Defs),
    io:format("remote tree : '~p'\n", [RemoteTree]).
复制代码

 

 

 

--The End--

目录
相关文章
|
开发工具 对象存储
阿里云OSS SDK上传文件报错:String contains non ISO-8859-1 code point
阿里云OSS SDK上传文件报错:String contains non ISO-8859-1 code point
611 0
学习:erlang的term反序列化,string转换为term
一、 string_to_term(String) ->    case erl_scan:string(String++".") of        {ok, Tokens, _} ->            case erl_parse:parse_term(Tokens) of       ...
851 0
|
2月前
|
Java 索引
java基础(13)String类
本文介绍了Java中String类的多种操作方法,包括字符串拼接、获取长度、去除空格、替换、截取、分割、比较和查找字符等。
36 0
java基础(13)String类
|
24天前
|
Java
【编程基础知识】(讲解+示例实战)方法参数的传递机制(值传递及地址传递)以及String类的对象的不可变性
本文深入探讨了Java中方法参数的传递机制,包括值传递和引用传递的区别,以及String类对象的不可变性。通过详细讲解和示例代码,帮助读者理解参数传递的内部原理,并掌握在实际编程中正确处理参数传递的方法。关键词:Java, 方法参数传递, 值传递, 引用传递, String不可变性。
46 1
【编程基础知识】(讲解+示例实战)方法参数的传递机制(值传递及地址传递)以及String类的对象的不可变性
|
20天前
|
安全 Java 测试技术
Java零基础-StringBuffer 类详解
【10月更文挑战第9天】Java零基础教学篇,手把手实践教学!
21 2
|
23天前
|
存储 安全 C++
【C++打怪之路Lv8】-- string类
【C++打怪之路Lv8】-- string类
17 1
|
26天前
|
数据可视化 Java
让星星月亮告诉你,通过反射创建类的实例对象,并通过Unsafe theUnsafe来修改实例对象的私有的String类型的成员属性的值
本文介绍了如何使用 Unsafe 类通过反射机制修改对象的私有属性值。主要包括: 1. 获取 Unsafe 的 theUnsafe 属性:通过反射获取 Unsafe类的私有静态属性theUnsafe,并放开其访问权限,以便后续操作 2. 利用反射创建 User 类的实例对象:通过反射创建User类的实例对象,并定义预期值 3. 利用反射获取实例对象的name属性并修改:通过反射获取 User类实例对象的私有属性name,使用 Unsafe`的compareAndSwapObject方法直接在内存地址上修改属性值 核心代码展示了详细的步骤和逻辑,确保了对私有属性的修改不受 JVM 访问权限的限制
48 4
|
2月前
|
安全 Java
String类-知识回顾①
这篇文章回顾了Java中String类的相关知识点,包括`==`操作符和`equals()`方法的区别、String类对象的不可变性及其好处、String常量池的概念,以及String对象的加法操作。文章通过代码示例详细解释了这些概念,并探讨了使用String常量池时的一些行为。
String类-知识回顾①
|
1月前
|
存储 安全 Java
【一步一步了解Java系列】:认识String类
【一步一步了解Java系列】:认识String类
24 2
|
1月前
|
安全 C语言 C++
【C++篇】探寻C++ STL之美:从string类的基础到高级操作的全面解析
【C++篇】探寻C++ STL之美:从string类的基础到高级操作的全面解析
33 4

热门文章

最新文章