DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in response to the amendments filed on 06/12/2026. Wherein, Claims 1, 7, 12, 13, 15, and 16 are amended. Claims 1-20 are rejected.
Response to Arguments
Applicant’s arguments, see REMARKS, filed 06/12/2026, with respect to the double patenting rejections have been fully considered and are persuasive. Therefore, the previous double patenting rejections have been withdrawn.
Applicant's arguments, with respect to the rejection of claims 1-20, under 35 USC § 102 have been fully considered and are persuasive. Therefore, the previous rejections under 35 USC § 102 have been withdrawn. However, a new rejection is presented below in view of Park et al.
With respect to the rejection of claims 1, 15, and 16, the Applicant states:
The Examiner's attention is directed to the fact that Marshall fails to disclose a method, non-transitory computer-readable medium, or apparatus comprising a processing system including at least one processor of an autonomous aerial vehicle determining informational data to present for at least one person, capturing at least a first image via at least one imaging sensor of the autonomous aerial vehicle in a vicinity of the at least one person, determining a first position of a plurality of positions and a first orientation of a plurality of orientations of the at least one person, wherein the plurality of positions is determined from location information from a network-based system that maintains location information of a user device associated with the at least one person, and wherein the plurality of orientations is determined from the at least the first image and from orientation information from the user device associated with the at least one person, identifying, based upon the first position and the first orientation of the at least one person, a plurality of candidate projection areas, each candidate projection area comprising a contiguous area within the at least the first image that is deemed to be visible to the at least one person in accordance with the first position and the first orientation, selecting one of the plurality of candidate projection areas as a projection zone in accordance with a plurality of factors, wherein the plurality of factors comprises: a number of people to whom the one of the plurality of candidate projection areas is deemed to be visible in accordance with the first position and the first orientation and a measure of at least one surface characteristic of the one of the plurality of candidate projection areas, and projecting the informational data on the projection zone via a projector, as positively claimed by independent claims 1, 15, and 16.
The Applicant specifically argues:
It is first noted that the Office Action refers to Marshall paragraph [0024] in connection with the claim features "determining a first position of a plurality of positions and a first orientation of a plurality of orientations of the at least one person," paragraphs [0025] and [0042] in connection with "wherein the plurality of positions is determined from location information from a network-based system that maintains location information of a user device associated with the at least one person," and paragraph [0040] in connection with "wherein the plurality of orientations is determined from the at least the first image." (See Office Action, p. 14-15).
In reply, it is respectfully submitted that the cited aspects of Marshall simply describe that "drones include one or more sensors to detect environmental patterns, movement patters or other factors" (See Marshall, para. [0024]), that "drones may receive information from other devices in the loT network that provides pertinent information" such as "profile and preference information that can be leveraged to determine what content to display" (See Marshall, para. [0025]), and "computer vision" may be used "to identify and select suitable surfaces to project an image" such as "a planar surface" with "a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image" (See Marshall, para. [0040]). It is noted that paragraph [0044] describes a module that "may include sensors and devices for emotion/engagement detection" that can "monitor crowds and detect if the user(s) are looking at the digital signage and also if the user(s) are smiling, are passive, etc." However, there does not appear to be any further description as to what these "sensors and devices" may comprise.
The relevant portion of previously presented claim 1 recites: “…determining a first position of a plurality of positions and a first orientation of a plurality of orientations of the at least one person, wherein the plurality of positions is determined from location information from a network-based system that maintains location information of a user device associated with the at least one person and wherein the plurality of orientations is determined from the at least the first image…”.
Marshall teaches that the drones track the audience movements and traffic patterns in order to determine what surfaces the display should be placed. Specifically, Marshall is trying to choose a surface that will be easily seen by the people in the crowd, i.e., in their view. Examiner agrees with the Applicant that Marshall’s “drones include one or more sensors to detect the environment patterns, movement patterns or other factors.” What the Applicant has left out of their description of Marshall is the context of “movement patterns”. As cited in ¶ [0024], the drones are tracking audience movements and traffic patterns as they change. By tracking the audience movement and traffic patterns the system is determining a position of a plurality of positions, i.e., multiple people in an audience, and the orientations of those people, i.e., what way they are moving in “traffic”.
Therefore, Marshall teaches “determining a first position of a plurality of positions of a first orientation of a plurality of orientations of the at least one person”.
Marshall later teaches that the drones are part of an IoT network and that through this network they receive information from the other devices of the network, e.g., wearable devices of the people in the audience. (¶ [0025]) In ¶ [0045], Marshall teaches that the wearable IoT to obtain data that can be leveraged for multiple functionalities. One of the “add-on functionalities” is determining, from the received data, the crowd’s contextual elements such as position, location of people in the crowd, and interests and current conversations as well as time availability.
Therefore, Marshall teaches “wherein the plurality of positions is determined from location information from a network-based system that maintains location information of a user device associated with at least one person”.
Marshall further teaches that the system uses a computer vision module, i.e., a module that detects and analyzes an image, to determine appropriate surfaces to project onto. (¶ [0040]) As provided in ¶ [0024], with respect to traffic direction, and further provided in ¶ [0042], the computer vision module is configured to determine if the audience is looking at the digital signage.
Therefore, Marshall teaches “and wherein the plurality of orientations is determined from the at least the first image”.
For the above reasons, the Examiner cordially disagrees with the Applicant’s argument that Marshall fails to teach “determining a first position of a plurality of positions and a first orientation of a plurality of orientations of the at least one person, wherein the plurality of positions is determined from location information from a network-based system that maintains location information of a user device associated with the at least one person and wherein the plurality of orientations is determined from the at least the first image”.
In this regard, it is further noted that the Office Action refers to Marshall paragraph [0040] in connection with "identifying, based upon the first position and the first orientation of the at least one person, a plurality of candidate projection areas, each candidate projection area comprising a contiguous area within the at least the first image that is deemed to be visible to the at least one person in accordance with the first position and the first orientation." (See Office Action, p. 15- 16). However, as indicated above, the cited aspect of Marshall merely states "computer vision" may be used "to identify and select suitable surfaces to project an image" such as "a planar surface" with "a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image," but does not describe that such actions involve identifying candidate projection areas deemed to be visible to the at least one person and more specifically in accordance with the first position and the first orientation.
As provided above, Marshall teaches that the system identifies a suitable surface to project onto. (¶ [0040]) Marshall discusses in ¶ [0024] and ¶ [0042], that the system monitors the audience, their traffic direction, and what they can see. Marshall is performing these functions to do determine a location which is in the view of the audience. As described in ¶ [0048] and shown in previously cited Fig. 5, “In FIG. 5, drone 520 is projecting an ad on bridge post 510. Drone 520 may select bridge post 510 due to its strategic location proximal walk path 540 and the ad's visibility to audience 530. In one embodiment, drone 520 may adjust its position relative to bridge post 210 so that the ad is visible from audience 530 located further away. To this end, drone 520 may use one or more optical train to size the ad or it may move closer or further away from bridge post 510.” (Emphasis added)
In summary, Marshall is gathering data on an audience and its members, it is tracking what way they are orientated and where they are at as well as if they are moving. Marshall is then determining a suitable surface to project onto so that the projected image may be visible to the audience, i.e., based on their location and orientation.
Therefore, Marshall teaches “identifying, based upon the first position and the first orientation of the at least one person, a plurality of candidate projection areas, each candidate projection area comprising a contiguous area within the at least the first image that is deemed to be visible to the at least one person in accordance with the first position and the first orientation.”
For these reasons the Examiner finds the above argument unpersuasive.
In addition, the Office Action further refers to Marshall paragraphs [0040] and [0046] in connection with "selecting one of the plurality of candidate projection areas as a projection zone in accordance with a plurality of factors, wherein the plurality of factors comprises: a number of people to whom the one of the plurality of candidate projection areas is deemed to be visible in accordance with the first position and the first orientation." (See Office Action, p. 16). However, paragraph [0046] simply states "the module may store crowd profiles (e.g., age, gender, crowd size, etc.) based on current or past events or occurrences. This information may be used for targeted displays and advertisement selection," which adds nothing to paragraph [0040].
As provided above by the Applicant, Marshall teaches “Module 460 may be configured to determine audience or crowd Profile. Here, the module may store crowd profiles (e.g., age, gender, crowd size, etc.) based on current or past events or occurrences.” (¶ [0046], emphasis added). Module 460 is part of system 400, which is used to determine what surface and where on that surface to project an image. Module 460 is used to determine crowd size, which is the same as the instant application’s limitation including “a number of people”.
In summary, Marshall discloses determining a suitable project surface based on data gathered by the modules of system 400. These modules include at least determining the location of the audience, their direction of visibility (orientation), and a size of a crowd. Based on the gathered data, the system then selects the surface that will be seen by the determined audience.
Therefore, Marshall teaches “selecting one of the plurality of candidate projection areas as a projection zone in accordance with a plurality of factors, wherein the plurality of factors comprises: a number of people to whom the one of the plurality of candidate projection areas is deemed to be visible in accordance with the first position and the first orientation.”
For these reasons the Examiner finds the above argument unpersuasive.
In summary, the cited aspects of Marshall do not first determine a first
orientation of a plurality of orientations of at least one person such that a plurality of candidate projection areas may be identified based upon the first position and the first orientation of the at least one person where each candidate projection area comprises a contiguous area within the at least the first image that is deemed to be visible to the at least one person in accordance with the first position and the first orientation and such that one of the plurality of candidate projection areas may be selected as a projection zone in accordance with a plurality of factors comprising a number of people to whom the one of the plurality of candidate projection areas is deemed to be visible in accordance with the first position and the first orientation and a measure of at least one surface characteristic of the one of the plurality of candidate projection areas. As such, Marshall fails to anticipate the present independent claims 1, 15, and 16.
The Examiner finds these arguments unpersuasive for the reasons provided above.
Notwithstanding, in the sole interest of advancing prosecution of the present application, and to further foreclose the Examiner's particular interpretations set forth in the Office Action, the independent claims are amended with support found in the Specification at least at paragraph [0032] to clarify and to additionally recite that the plurality of orientations is determined from the at least the first image and from orientation information from the user device associated with the at least one person.
For instance, as described in the present disclosure, user devices may report direction/orientation information to an AAV and/or to a wireless access network(s), which may forward the direction/orientation information to the AAV. (See, e.g., Specification, para. [0032]). In one example, an AAV may also track orientations (as well as positions) via LiDAR sensing or the like, e.g., for confirmation. (See, e.g., Specification, para. [0032]). It is respectfully submitted that Marshall simply does not describe any way in which user/user device orientation information is obtained, and in any case does not describe a process in which orientation information is confirmed via two sources: e.g., from user devices and from image information. Accordingly, Marshall remains further deficient with respect to at least these additional features, as now recited in the present independent claims 1, 15, and 16.
Marshall discloses the leveraging of mobile/wearable devices of the audience members as well as determining orientations from images and the location of members of an audience. Marshall does not explicitly teach that orientation information is obtained from the user device associated with at least one person.
For this reason, the previous rejection under 35 USC § 102 is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Marshall et al. (US 2019/0051224 A1, “Marshall”) in view of Park et al. (US 2019/0087145 A1, “Park”).
Regarding claims 1, 15 and 16, Marshall discloses systems, methods and apparatus for self-coordinated drone based digital signage and teaches:
An apparatus comprising: (the invention is directed towards drones, i.e., an apparatus – See at least ¶ [0019])
a processing system including at least one processor of an autonomous aerial vehicle; and (As described below, each drone may include one or more processing circuitries and memory circuitries to enable flight pattern and positioning of the drone to project the desired signage – See at least ¶ [0019])
a computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising: (Examples, as described herein, may include, or may operate on, logic or a number of components, modules, or mechanisms. Modules are tangible entities (e.g., hardware) capable of performing specified operations when operating. A module includes hardware. In an example, the hardware may be specifically configured to carry out a specific operation (e.g., hardwired). In an example, the hardware may include configurable execution units (e.g., transistors, circuits, etc.) and a computer readable medium containing instructions, where the instructions configure the execution units to carry out a specific operation when in operation – See at least ¶ [0056])
determining informational data to present for at least one person; (Where a group of drones are used to display a signage, one of the drone can take the role of quality assessment. It can take pictures/video from the audience level and determine if the sign is oriented properly, is bright enough and is gathering sufficient interest. A drone may analyze the audience's interest level and adjust content accordingly. The drones may also continuously analyze the audience (e.g., gender, age or other features.) and tailor content accordingly – See at least ¶ [0017])
capturing at least a first image via at least one imaging sensor of the autonomous aerial vehicle in a vicinity of the at least one person; (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040])
determining a first position of a plurality of positions and a first orientation of a plurality of orientations of the at least one person, (The drones may be configured to reorganize and continue to display on various other surfaces as the audience moves or as the traffic pattern of audience changes. In still other embodiments, the drones may consider the crowd's aggregated context and profile in order to determine the digital signage to be displayed (i.e., what product or service that can be of interest of the majority of people) and how (i.e., business or consumer focus, specific events, etc.) To this end, the drones include one or more sensors to detect environmental patterns, movement patterns or other factors – See at least ¶ [0024]) wherein the plurality of positions is determined from location information from a network-based system that maintains location information of a user device associated with the at least one person, (In certain embodiments, the drones may be part of an Internet of Things (IoT) network. In this manner, the drones may receive information from other devices in the IoT network that provides pertinent information. An IoT device may include wearable wireless devices. Wearable devices can complement the system by monitoring each user's response and attention to the signage. Mobile devices may contain profile and preference information that can be leveraged to determine what content to display. Wearable or mobile devices can complement the system by providing the user with an easy to use method to capture the signage for future consumption, for example, by pointing the device in the direction of the signage. In this manner, the system can keep track of prior interactions with the digital signage to create and update each user profile and preferences – See at least ¶ [0025] Module 430 may include sensors and devices for emotion/engagement detection of audience. Such modules can monitor crowds and detect if the user (s) are looking at the digital signage and also if the user(s) are smiling, are passive, etc. wearable or mobile technologies (e.g., IoT) can be leveraged for emotion response detection. An optional module (or an add-on functionality) may be included to provide context detection. Here, the system may monitor the crowd's contextual elements, such as position, location of people in a crowd, interests and current conversations (if users have opted-in), time availability etc. An optional module (or an add-on functionality) may be included to keep track of previous interaction of each person with a digital signage and effectiveness of the content (history module ) – See at least ¶ [0042]) and wherein the plurality of orientations is determined from the at least the first image (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040]; Examiner notes that the “appropriate projection surface” includes a surface that is in the direct line of sight of the audience, e.g., the orientation.) and from [] information from the users device associated with the at least one person; (Mobile devices may contain profile and preference information that can be leveraged to determine what content to display – See at least ¶ [0025])
identifying, based upon the first position and the first orientation of the at least one person, a plurality of candidate projection areas, each candidate projection area comprising a contiguous area within the at least the first image that is deemed to be visible to the at least one person in accordance with the first position and the first orientation; (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040]; As shown in Fig. 1 and 5 the projection areas are contiguous.)
selecting one of the plurality of candidate projection areas as a projection zone in accordance with a plurality of factors, wherein the plurality of factors comprises: (Module 420 may include computer vision to identify and select suitable surfaces to project an image – See at least ¶ [0040])
a number of people to whom the one of the plurality of candidate projection areas is deemed to be visible in accordance with the first position and the first orientation (Module 460 may be configured to determine audience or crowd profile. Here, the module may store crowd profiles (e.g., age, gender, crowd size, etc.) based on current or past events or occurrences. This information may be used for targeted displays and advertisement selection – See at least ¶ [0046]) and a measure of at least one surface characteristic of the one of the plurality of candidate projection areas; and (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040])
projecting the informational data on the projection zone via a projector. (Each of Drone 102, 104, 106 and 108 may be configured to be self-guided and position itself with respect to building 100 and the other drones to project the desired signage – See at least ¶ [0019])
Marshall does not explicitly teach wherein the plurality of orientations is determined from the at least one image and from orientation information from the user’s device associated with at least one person. However, Park discloses a contextual presentation system and teaches:
[] wherein the plurality of orientations is determined from the at least the first image (The camera 2304 may be a machine vision camera. The mobile UAS 2302 may fly to approach an audience. Similarly, recognition of landmarks, terrain mapping data may be used as well – See at least ¶ [0122]) and from orientation information from the users device associated with the at least one person; (FIG. 24 is a top view of a mobile UAS 2400 communicating with a server 2402 (such as the centralized server 102) on the ground to determine the averaged central position of each group of audiences measured by sum of each position of audience using each individual's smart phone application GPS reading. The audience point may be obtained from preprogrammed values, or it may be dynamically updated from the server 2402 on the ground or onboard by collecting the position of each individual in the audience, who has an application installed on her smartphones (or other devices such as wearable devices). The application may update the position and desired contents, photos, or videos to be displayed on the display of the mobile UAS 2400. Accordingly, the mobile UAS 2400 may change its flight path and display contents received from the server 2402 – See at least ¶ [0123])
In summary, Marshall discloses choosing a projection area based on the position, orientation, movement, and context of the audience. Marshall further teaches determining the orientation of the crowd based on machine vision and additionally leveraging the audience members smart devices to determine other data about the members. Marshall does not explicitly teach that orientation data is obtained from the smart devices. However, Park discloses a contextual presentation system that presents data, via drones, to audiences in a wide variety of locations and circumstances. The presentation location is based on computer vision and position data of the users received from their mobile devices.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the instant application to have modified the systems, methods, and apparatus for self-coordinated drone based digital signage of Marshall to provide for the contextual presentation system, as taught in Park, to make the information presentation responsive to audience crowd densities and types. (At Park ¶ [0048])
Regarding claims 2 and 17, Marshall further teaches:
identifying, based upon the first position and the first orientation of the at least one person, a projection search space, the projection search space comprising an area that is captured in the at least the first image, that is within range of the projector of the autonomous aerial vehicle, and that is deemed to be visible to the at least one person. (In the exemplary embodiment of FIG. 1, drones 102, 104, 106 and 108 may scan building 100 to identify one or more portions of building to display images 103, 105, 106, and 107. The scanning may define geospatial scanning so as to identify location which will he most likely viewed by an audience, such as passer byes. For example, drones 102, 104, 106 and 108 may scout for locations outside of a baseball stadium and identify building 100 in the stadium exit path such that the patrons may have a clear line of sight (LOS) to the displayed images upon exiting the venue. As stated, one or more drone may reposition itself by changing location or projection angle to thereby readjust the image. Image readjustment may be done to accommodate the audience's LOS or changing environmental factors – See at least ¶ [0021] and [0048])
Regarding claims 3 and 18, Marshall further teaches:
determining an orientation to display the informational data in the projection zone, based upon a location of the projection zone, the first position, and the first orientation. (In the exemplary embodiment of FIG. 1, drones 102, 104, 106 and 108 may scan building 100 to identify one or more portions of building to display images 103, 105, 106, and 107. The scanning may define geospatial scanning so as to identify location which will he most likely viewed by an audience, such as passer byes. For example, drones 102, 104, 106 and 108 may scout for locations outside of a baseball stadium and identify building 100 in the stadium exit path such that the patrons may have a clear line of sight (LOS) to the displayed images upon exiting the venue. As stated, one or more drone may reposition itself by changing location or projection angle to thereby readjust the image. Image readjustment may be done to accommodate the audience's LOS or changing environmental factors – See at least ¶ [0021])
Regarding claims 4 and 19, Marshall further teaches:
wherein the informational data comprises directional information. (The projector drone(s) and the plurality of surface-forming drones may move or change presentation angle to remain well within the audiences' line of sight. The drones may also output 3D audio sound by spatially positioning around an audience and emit the sound to produce 3D sounds. In one embodiment, the drones may project a directional symbol like an arrow to draw attention to something of interest such as a building or billboard – See at least ¶ [0016])
Regarding claims 5 and 20, Marshall further teaches:
wherein the informational data comprises a safety message. (FIG. 6 shows an exemplary embodiment in which drone projection is used to magnify a structural defect according to one embodiment of the disclosure. Specifically, the disclosed embodiment shows an exemplary embodiment in which one or more drone is used to survey an area or a defect. In FIG. 6, bridge post 630 includes a crack which may not be readily visible from vantage point of audience 530. Here, drone 620 is used to project an augmented display of the bridge crack. In one application of this embodiment, the content (i.e., image of the defect) may be projected at a desired location after an initial image of the defect is obtained. The system may communicate with audience 630 to position themselves appropriately to view the defect as it appears (in smaller size) on the bridge – See at least ¶ [0049]; Examiner notes that by projecting the structural deficiency, i.e., information related to safety, the drone is creating a safety message.)
Regarding claim 6, Marshall further teaches:
wherein the at least one person is among a group of people, and wherein the group of people is detected by the processing system via at least one of: the at least one imaging sensor or via at least one other sensor of the autonomous aerial vehicle. (Module 430 may include sensors and devices for emotion/engagement detection of audience. Such modules can monitor crowds and detect if the user (s) are looking at the digital signage and also if the user(s) are smiling, are passive, etc. wearable or mobile technologies (e.g., IoT) can be leveraged for emotion response detection. An optional module (or an add-on functionality) may be included to provide context detection. Here, the system may monitor the crowd's contextual elements, such as position, location of people in a crowd, interests and current conversations (if users have opted-in), time availability etc. An optional module (or an add-on functionality) may be included to keep track of previous interaction of each person with a digital signage and effectiveness of the content (history module) – See at least ¶ [0042])
Regarding claim 7, Marshall further teaches:
wherein the identifying the plurality of candidate projection areas is based upon a second plurality of positions and a second plurality of orientations of people within the group, and wherein each candidate projection area is deemed to be visible to at least one of the people within the group. (In one embodiment, the signage is created ad hoc and based on potential audience's movement pattern. In another embodiment, the ad hoc surface is created and located to be in direct line of sight of a potential audience. In still another embodiment, the surface is configured to move or change direction as the audience moves or changes its line of sight. In yet another embodiment, the disclosure relates to amplifying projection of a visual ad using multiple projector drones – See at least ¶ [0018])
Regarding claim 8, Marshall further teaches:
wherein the plurality of factors further comprises: a size of the one of the plurality of projection areas. (The display content may be adjusted based on aggregated needs, context and profile of the audience. If in a scenario where people are walking, then the content can be more detailed. If projection is along highway where cars are zooming by, then the system may make the projection size larger and with less details – See at least ¶ [0015])
Regarding claim 9, Marshall further teaches:
wherein the plurality of factors further comprises: a range of the one of the plurality of candidate projection areas with respect to the projector of the autonomous aerial vehicle. (Drone 520 may select bridge post 510 due to its strategic location proximal walk path 540 and the ad's visibility to audience 530. In one embodiment, drone 520 may adjust its position relative to bridge post 210 so that the ad is visible from audience 530 located further away. To this end, drone 520 may use one or more optical train to size the ad or it may move closer or further away from bridge post 510 – See at least ¶ [0048])
Regarding claim 10, Marshall further teaches:
wherein the plurality of candidate projection areas includes at least one of: a ground surface candidate projection area; or
a non-ground surface candidate projection area. (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040])
Regarding claim 11, Marshall further teaches:
capturing at least a second image via the at least one imaging sensor of the autonomous aerial vehicle; (As shown in Fig. 3, the image projecting process is continual and dynamic. Meaning that there will be a second image from the image sensor when user interest wane or there is a location change.)
determining a second position and a second orientation of the at least one person; (The drones may be configured to reorganize and continue to display on various other surfaces as the audience moves or as the traffic pattern of audience changes. In still other embodiments, the drones may consider the crowd's aggregated context and profile in order to determine the digital signage to be displayed (i.e., what product or service that can be of interest of the majority of people) and how (i.e., business or consumer focus, specific events, etc.) To this end, the drones include one or more sensors to detect environmental patterns, movement patterns or other factors – See at least ¶ [0024])
identifying, based upon the second position and the second orientation of the at least one person, a second plurality of candidate projection areas deemed to be visible to the at least one person; (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040])
selecting one of the second plurality of candidate projection areas as a second projection zone; and (Module 420 may include computer vision to identify and select suitable surfaces to project an image – See at least ¶ [0040])
projecting the informational data on the second projection zone. (Each of Drone 102, 104, 106 and 108 may be configured to be self-guided and position itself with respect to building 100 and the other drones to project the desired signage – See at least ¶ [0019])
Regarding claim 12, Marshall further teaches:
wherein the second projection zone is of at least one of: a different size from a size of the projection zone; (Module 420 may include computer vision to identify and select suitable surfaces to project an image. Thus, an exemplary drone may seek to find an appropriate projection surface in the crowd's vicinity. An optimal surface may be a planar surface and have a lighter material property or it could be any arbitrary objects like tree, buildings or dense clouds to receive a project image – See at least ¶ [0040])
a different shape from a shape of the projection zone; or (Examiner notes that optimal surfaces, i.e., projection surfaces that can change over time, include buildings, trees, and clouds. These objects have different shapes from one another – See at least ¶ [0040])
a different orientation from an orientation of the projection zone. (Throughout the display process, the drones may self-adjust or rearrange to provide clear LOS and to accommodate environmental changes. For example, a master drone may dictate position changes, tilt angle or other flight In requirements to slave drones to maintain a clear LOS to the audience – See at least ¶ [0034])
Regarding claim 13, Marshall further teaches:
further comprising: detecting at least one item of interest in the projection zone; and (FIG. 6 shows an exemplary embodiment in which drone projection is used to magnify a structural defect according to one embodiment of the disclosure. Specifically, the disclosed embodiment shows an exemplary embodiment in which one or more drone is used to survey an area or a defect. In FIG. 6, bridge post 630 includes a crack which may not be readily visible from vantage point of audience 530. Here, drone 620 is used to project an augmented display of the bridge crack. In one application of this embodiment, the content (i.e., image of the defect) may be projected at a desired location after an initial image of the defect is obtained. The system may communicate with audience 630 to position themselves appropriately to view the defect as it appears (in smaller size) on the bridge – See at least ¶ [0049])
illuminating the at least one item of interest via at least one lighting unit of the autonomous aerial vehicle. (Here, drone 620 is used to project an augmented display of the bridge crack. In one application of this embodiment, the content (i.e., image of the defect) may be projected at a desired location after an initial image of the defect is obtained. The system may communicate with audience 630 to position themselves appropriately to view the defect as it appears (in smaller size) on the bridge – See at least ¶ [0049]; Examiner notes that a projector is a lighting unit)
Regarding claim 14, Marshall further teaches
wherein the detecting the at least one item of interest comprises detecting an outline of the at least one item of interest, and wherein the illuminating comprises highlighting the outline of the at least one item of interest via the at least one lighting unit. (As shown in Fig. 6 the crack is an “outline” and by projecting the augmented display of the crack the drone is highlighting the outline of the crack – See at least Fig. 6)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Pearce (US 2018/0079463 A1) which discloses bicycle safety exclusion zone systems and teaches projecting a safety message onto the ground or a vehicle.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHASE L COOLEY whose telephone number is (303)297-4355. The examiner can normally be reached Monday-Thursday 7-5MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at 571-270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.L.C./Examiner, Art Unit 3662
/ANISS CHAD/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights