[幻灯片] Erlang/OTP 两三事

Erlang/OTP 两三事

gen_server

提纲

• gen_server 概述

• gen_server 实例

• gen_server 源码

• gen_server 陷阱

• gen_server 技巧

gen_server 概述

gen_server 概述

gen_server 概述

提纲

• gen_server 概述(X)

• gen_server 实例

• gen_server 源码

• gen_server 陷阱

• gen_server 技巧

gen_server 实例

%% interface start_link() ->     gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).  %% callback init([]) ->     {ok, #state{}, ?HIBERNATE_TIMEOUT}.

gen_server 实例

%% interface  %% gen_server client need a new ets table new_ets() ->  gen_server:call(?SERVER, create_ets_table).   %% callback %% gen_server server process will execute new op and keep the ets table alive handle_call(create_ets_table, _From, State) ->  EtsTable = ets:new(ets_table, [public]),  {reply, EtsTable, State, ?HIBERNATE_TIMEOUT};

gen_server 实例

%% interface %% gen_server client is very lazy, would not execute logic by itself check_method_valid(MethodName) ->  gen_server:call(?SERVER, {if_method_valid, MethodName}).  %% callback %% then gen_server server help client handle_call({if_method_valid, MethodName}, _From, State) ->  case MethodName of   'call' -> {reply, true, State, ?HIBERNATE_TIMEOUT};   'cast' -> {reply, true, State, ?HIBERNATE_TIMEOUT};   _      -> {reply, false, State, ?HIBERNATE_TIMEOUT}  end;

gen_server 实例

%% interface %% gen_server client has no resource which server had, so has to call server push_msg(Target, Msg) when erlang:is_pid(Target) ->  gen_server:cast(?SERVER, {push_msg, Target, Msg}).  %% callback %% gen_server server do handle_cast({push_msg, Target, Msg}, State) ->  erlang:send(Target, Msg),  {noreply, State, ?HIBERNATE_TIMEOUT};

提纲

• gen_server 概述(X)

• gen_server 实例(X)

• gen_server 源码

• gen_server 陷阱

• gen_server 技巧

gen_server 源码

gen_server 源码

gen_server 源码

start(GenMod, LinkP, Name, Mod, Args, Options)     GenMod  :: gen_server | gen_fsm     LinkP   :: nolink     | link     Name    :: {local, atom()} | {global, term()} | {via, atom(), term()} %% 指定的name     Mod     :: atom() %% 就是使用了gen_server behaviour 的模块名     Args    :: term() %% Mod 模块 init func 的初始参数     Options :: [{timeout, Timeout} | {debug, [Flag]} | {spawn_opt, OptionList}]         %% 这个参数是用来指定创建进程的参数     Flag    :: trace | log | {logfile, File} | statistics | debug

gen_server 源码

init_it(Starter, self, Name, Mod, Args, Options) ->  %% 注意, 如果使用nolink start 时, Parent 就是自己  init_it(Starter, self(), Name, Mod, Args, Options); init_it(Starter, Parent, Name0, Mod, Args, Options) ->  Name = name(Name0),  Debug = debug_options(Name, Options),  case catch Mod:init(Args) of   {ok, State} ->    proc_lib:init_ack(Starter, {ok, self()}), %% 向调用者返回结果     loop(Parent, Name, State, Mod, infinity, Debug);   {ok, State, Timeout} ->    proc_lib:init_ack(Starter, {ok, self()}),       loop(Parent, Name, State, Mod, Timeout, Debug);   ………… 

gen_server 源码

gen_server 的主体module start :

gen_server 源码

执行gen_server:init_it/6 之后, 会进入loop :

gen_server 源码

触发gen_server terminate 的场景 :

gen_server 源码

long-lived process

In general, Erlang processes are expected toshort-lived and have small amounts of live data. When there is not enough free memory in the heap for a process, it is garbage collected, and if less memory can be freed than required it grows.Each process’ heap is garbage collected independently, when a process exits, its memory is simply reclaimed. 

gen_server 源码

hibernate

通过整理进程的stack,回收进程的heap 来达到回收内存节省资源的效果

• 用于某进程可能会在未来短期内空闲时
• hibernate 会清空进程的stack 然后进程GC

gen_server 源码

hibernate on mochiweb

This sounds reasonable -  let's try hibernating after every message and see what happens .

gen_server 源码

hibernate 缺点

Note that emptying the call stack means that any surrounding catch is removed and has to be re-inserted after hibernation. 

需要CPU , 可能带来不必要的empty-reinsert 操作.

gen_server 源码

hibernate on ejabberd

Ejabberd 对hibernate 的使用比较谨慎, 只有在进程未收到任何信息一段时间后, 才使用hibernate .

handle_call(create_ets_table, _From, State) ->     EtsTable = ets:new(ets_table, [public]),     {reply, EtsTable, State, ?HIBERNATE_TIMEOUT};

handle_info(timeout, State) ->     proc_lib:hibernate(gen_server, enter_loop,                [?MODULE, [], State]),     {noreply, State, ?HIBERNATE_TIMEOUT};

gen_server 源码

hibernate on ejabberd

实现原理：

handle_msg({'$gen_call', From, Msg}, Parent, Name, State, Mod) -> Result = try_handle_call(Mod, Msg, From, State), case Result of  %% ...  {ok, {reply, Reply, NState, Time1}} ->   reply(From, Reply),   loop(Parent, Name, NState, Mod, Time1, []);  %% ...  {ok, {noreply, NState, Time1}} ->   loop(Parent, Name, NState, Mod, Time1, [])  %% ...   end; 

loop(Parent, Name, State, Mod, Time, Debug) ->     Msg = receive           Input ->             Input       after Time ->           timeout       end,     decode_msg(Msg, Parent, Name, State, Mod, Time, Debug, false).

gen_server 源码

hibernate on RabbitMQ

init(Q) ->     process_flag(trap_exit, true),     ?store_proc_name(Q#amqqueue.name),     {ok, init_state(Q#amqqueue{pid = self()}), hibernate,      {backoff, ?HIBERNATE_AFTER_MIN, ?HIBERNATE_AFTER_MIN, ?DESIRED_HIBERNATE},     ?MODULE}.

提纲

• gen_server 概述(X)

• gen_server 实例(X)

• gen_server 源码(X)

• gen_server 陷阱

• gen_server 技巧

gen_server 陷阱

• 资源不易回收 (X)
• 单进程瓶颈

每个Erlang Process 默认 获得的资源是 相同 的

gen_server 陷阱

• 单进程瓶颈  - noblock call

gen_server 陷阱

• 单进程瓶颈  - noblock call  - rpc

handle_call_call(Mod, Fun, Args, Gleader, To, S) -> RpcServer = self(), %% Spawn not to block the rpc server. {Caller,_} =  erlang:spawn_monitor(    fun () ->      set_group_leader(Gleader),      Reply =        %% in case some sucker rex'es        %% something that throws       case catch apply(Mod, Fun, Args) of        {'EXIT', _} = Exit ->         {badrpc, Exit};        Result ->         Result       end,      RpcServer ! {self(), {reply, Reply}}    end), {noreply, gb_trees:insert(Caller, To, S)}. 

handle client call via spawn a new process for temporary

gen_server 陷阱

• 单进程瓶颈  - noblock call  - rpc

handle_info({Caller, {reply, Reply}}, S) -> case gb_trees:lookup(Caller, S) of  {value, To} ->   receive    {'DOWN', _, process, Caller, _} ->      gen_server:reply(To, Reply),     {noreply, gb_trees:delete(Caller, S)}   end;  none ->   {noreply, S} end; 

return the result to client by `gen_server:reply/2`

gen_server 陷阱

• 单进程瓶颈  - noblock call  - net_kernel

handle_call({spawn_opt,M,F,A,O,L,Gleader},{From,Tag},State) when is_pid(From) ->  do_spawn([L,{From,Tag},M,F,A,Gleader],O,State); do_spawn(SpawnFuncArgs, SpawnOpts, State) ->  [_,From|_] = SpawnFuncArgs,  case catch spawn_opt(?MODULE, spawn_func, SpawnFuncArgs, SpawnOpts) of    {'EXIT', {Reason,_}} ->    async_reply({reply, {'EXIT', {Reason,[]}}, State}, From);    {'EXIT', Reason} ->     async_reply({reply, {'EXIT', {Reason,[]}}, State}, From);    _ ->     {noreply,State}  end. spawn_func(link,{From,Tag},M,F,A,Gleader) ->  link(From),  gen_server:reply({From,Tag},self()),  %% ahhh  group_leader(Gleader,self()),  apply(M,F,A); spawn_func(_,{From,Tag},M,F,A,Gleader) ->  gen_server:reply({From,Tag},self()),  %% ahhh  group_leader(Gleader,self()),  apply(M,F,A). 

gen_server 陷阱

• 单进程瓶颈  - pool

提纲

• gen_server 概述(X)

• gen_server 实例(X)

• gen_server 源码(X)

• gen_server 陷阱(X)

• gen_server 技巧

gen_server 技巧

• 使用system msg debug
• 预分配合适的资源
• 适当干预资源回收
• 使用task process
• gen_server pool

提纲

• gen_server 概述(X)

• gen_server 实例(X)

• gen_server 源码(X)

• gen_server 陷阱(X)

• gen_server 技巧(X)

参考

Q&A