请问BaselineAgent是否具有_shoot_action的能力？

[No-9527149]

尊敬的开发者团队你们好，在/envs/JSBSim/configs/1v1/ShootMissile中，已经预先配置了HierarchyVsBaseline_nolimit.yaml && HierarchyVsBaseline.yaml 参数文件。

但是我在尝试使用这两份.yaml文件进行训练时，遇到了一个问题，具体的报错如下所示：

Process Process-1:
Traceback (most recent call last):
  File "/anaconda3/envs/jsb/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/anaconda3/envs/jsb/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/CloseAirCombat/envs/env_wrappers.py", line 231, in worker
    remote.send([step_env(env, action) for env, action in zip(envs, data)])
  File "/CloseAirCombat/envs/env_wrappers.py", line 231, in <listcomp>
    remote.send([step_env(env, action) for env, action in zip(envs, data)])
  File "/CloseAirCombat/envs/env_wrappers.py", line 208, in step_env
    obs, reward, done, info = env.step(action)
  File "/CloseAirCombat/envs/JSBSim/envs/env_base.py", line 154, in step
    a_action = self.task.normalize_action(self, agent_id, action[agent_id])
  File "/CloseAirCombat/envs/JSBSim/tasks/single_combat_with_missile_task.py", line 296, in normalize_action
    self._shoot_action[agent_id] = action[-1]
TypeError: 'NoneType' object is not subscriptable

经过我的分析，以1v1的情况为例，A0100的actions是通过/algorithms/utils/act.py获得的，在/envs/JSBSim/envs/env_base.py BaseEnv step()中通过self._unpack(action)后，按照agent_id和进程数量解析，而此时_unpack之后B0100的动作为None。

_unpack之后的actions，进入了task.normalize_action(self, agent_id, action[agent_id])，而对于HierarchicalSingleCombatShootTask.normalize_action函数:

def normalize_action(self, env, agent_id, action):
        """Convert high-level action into low-level action."""
        self._shoot_action[agent_id] = action[-1]
        return HierarchicalSingleCombatTask.normalize_action(
            self, env, agent_id, action[:-1].astype(np.int32)
        )

先根据agent_id获取action的最后一项，即shoot_actio，但此时agent_id为B0100对应的action为None，报错就出现在这里

之后才进入HierarchicalSingleCombatTask.normalize_action函数，对动作进行归一化：

def normalize_action(self, env, agent_id, action):
        """Convert high-level action into low-level action."""
        if self.use_baseline and agent_id in env.enm_ids:
            action = self.baseline_agent.get_action(env.agents[agent_id])
            return action
        else:
            # generate low-level input_obs
            raw_obs = self.get_obs(env, agent_id)
            input_obs = np.zeros(12)
            # (1) delta altitude/heading/velocity
            input_obs[0] = self.norm_delta_altitude[action[0]]
            input_obs[1] = self.norm_delta_heading[action[1]]
            input_obs[2] = self.norm_delta_velocity[action[2]]
            # (2) ego info
            input_obs[3:12] = raw_obs[:9]
            input_obs = np.expand_dims(input_obs, axis=0)
            # output low-level action
            _action, _rnn_states = self.low_level_policy(
                input_obs, self._inner_rnn_states[agent_id]
            )
            action = _action.detach().cpu().numpy().squeeze(0)
            self._inner_rnn_states[agent_id] = _rnn_states.detach().cpu().numpy()
            # normalize low-level action
            norm_act = np.zeros(4)
            norm_act[0] = action[0] / 20 - 1.0
            norm_act[1] = action[1] / 20 - 1.0
            norm_act[2] = action[2] / 20 - 1.0
            norm_act[3] = action[3] / 58 + 0.4
            return norm_act

此时才会根据use_baseline参数来获取BaselineAgent的动作。

目前看来存在一个矛盾：

1. 既然.yaml文件存放于/configs/1v1/ShootMissile任务中，那么HierarchyVsBaseline.yaml应当支持BaselineAgent，并且BaselineAgent.get_action得到的动作，可以提供shoot_action
2. 从目前的代码逻辑来看，并不支持BaselineAgent完成shoot_action

我想请教的问题是：baseline_model.pt是否支持生成包含shoot_action的能力？如果支持，那目前的代码逻辑应该如何修改？（比如调换HierarchicalSingleCombatShootTask.normalize_action中的上下两行？

希望能够得到各位开发者的回复，感谢！