Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-8, 10-11, and 19-24 are currently amended. No new claims have been added. No claims have been canceled. Claims 1-24 are currently pending for examination.
Response to Arguments
In response to the applicant’s arguments, pgs.6-9, the examiner has carefully considered these arguments. In consideration of the amended limitation; “generating, by an operating system and in response to detecting the feature associated with the tag, instructions for the application” does provide an improvement to the technological field of accessibility within video game systems. The generation of the instructions for the game application by the operating system provides an advantage over the standard methods for accessibility provision, such as those recited in Rainbolt (US 11654363 B1). As such, the amended claim limitations provide an improvement to the technological field of providing accessibility features within video games. Accordingly, the rejection of the instant application under 35 USC 101 is withdrawn.
In response to the applicant’s arguments, pg.10, that neither Rainbolt nor any of the other cited prior arts of record disclose the amended limitations of the independent claims, the examiner has carefully considered these arguments and concedes. As such, the new grounds of rejection rely upon Resnick (Bai (US 20240135973 A1) to disclose the amended limitations of the independent claims. Accordingly, the rejection of the instant application under 35 USC 102 and 35 USC 103 is maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 21, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1).
As per claim 1, Resnick discloses:
A method for enhancing accessibility for an application (“For instance, a user may be able to watch media content on a first client system 110-1 (e.g., television) and feel haptic effects on a second client system 110-2 (e.g., smartphone), where the first client system 110-1 and second client system 110-2 are synced in real-time. This type of haptics experience can increase accessibility for users with visual/auditory impairment.”, 0029)
a) associating, by a trained machine learning module, a feature with a tag; (“The haptic data 422 is an example representation of haptic data 220 or haptic data 222 described with respect to FIG. 2. Here, one or more haptic event tags (or keywords) (associated with haptic data 422) can be inserted into the timestamped series of metadata within the speech marks file 410.”, 0075 ; “The analysis tool 240 includes one or more ML models 212, which are generally configured to generate haptic data 220 for the media content 202.”, 0043; “To train the ML model 226, image segmentation may be performed to isolate different objects in one or more video frames (different from the video 206).”, 0036)
b) associating, by a trained machine learning module, the tag with an action (“The analysis tool 240 may access a database of predefined haptic effects for different types of events and may map a given feature 232 to one of the predefined haptic effects.”, 0046 ; “The analysis tool 240 includes one or more ML models 212, which are generally configured to generate haptic data 220 for the media content 202.”, 0043)
c) detecting, while causing a video stream generated by an application the feature associated with the tag (“In one embodiment, the extraction tool 210 uses the ML model 226 to determine (or detect) different types of motions that are going on in different scenes of the video 206.”, 0035 ; “In the depicted embodiment, the extraction tool 210 receives media content 202 (including audio 204, video 206, or audio 204 and video 206) and determines (or extracts) one or more features 232 of the media content 202, based on analysis of the media content 202. The extraction tool 210 can extract features 232 of the audio 204, features 232 of the video 206, or features 232 of the audio 204 and video 206.”, 0033 ; “The analysis tool 240 may access a database of predefined haptic effects for different types of events and may map a given feature 232 to one of the predefined haptic effects.”, 0046”)
d) triggering, by the application and in response to receiving the instructions, a device to implement the instructions based on the tag. (“As shown, the client system 110 includes a haptic engine 116, which is generally configured to trigger (or activate) the haptic device(s) 118, based on the haptic data. The haptic device(s) 118 may be located on the client system 110 or external to the client system 110. The haptic device(s) 118 can include one or more physical actuators, such as piezoelectric actuator(s), electromagnetic actuator(s), etc., speakers, touchscreen, etc. The haptic engine 116 can control one or more properties (or parameters) of the haptic device(s) 118, based on the haptic data, to generate one or more haptic responses or effects or patterns.”, 0027)
Resnick discloses the above limitations of claim 1, but does not explicitly disclose the generation of instructions being performed by an operating system.
However, Bai discloses:
generating, by an operating system and in response to detecting the feature associated with the tag, instructions for the application (“At block 2710, a transcript interface is caused to present a visualization of a transcript of a video. The visualization (i) stylizes a visual appearance of transcript text corresponding to a portion of the video where a feature was detected, or (ii) labels the transcript text with a textual representation of a detected class of the feature.”, 0171 ; “In some cases, the application is integrated into an operating system (e.g., as a service).”, 0071)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick with those of Bai in order to provide the system with a video ingestion tool which downsamples video content to facilitate faster processing (Bai, [0096]).
As per claim 21, it is a system claim (see Resnick: [0087], [0093]) comprising substantially the same limitations as claim 1, and as such, it is rejected for substantially the same reasons.
As per claim 23, it is a computer readable medium (see Resnick: [0090]) claim comprising substantially the same limitations as claim 1, and as such, it is rejected for substantially the same reasons.
Claims 2, 14, 16, 18, 22, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Rainbolt (US 11654363 B1).
As per claim 2, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose the association of a tag with a type of tag.
However, Rainbolt discloses:
associating feature with the tag comprises associating the tag with a type, and associating the tag with an action comprises associating the type with the instructions ("The tag may be one of a set of tags defined or provided for a game, in order to provide for simplified management of the game, wherein similar objects or elements can have the same tags, or types of tags, applied, which can help to also provide for more consistent gameplay if the same types of objects or elements have the same types of interactions associated with them.", col.3, lines 24-30)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick in view of Bai with those of Rainbolt in order to provide the system with the ability to set different collision actions for multiple items, individually or as a group, and to associate the functionality on the elements or groups in a simplified manner (Rainbolt, [col.3, lines 64-67]).
As per claim 14, Resnick in view of Bai fully discloses the limitations of claim 1, but does not explicitly disclose the feature including an asset.
However, Rainbolt discloses:
the feature includes an asset from the application. (" In one embodiment a rules system enables a developer to set up the various relationships and expressions and then save that information to an asset file. The file can then be saved to a gaming repository. When a developer then wants to create a new gameplay entity, in the same game or a different game, the developer can look at a list of available rules, tags, expressions, collision matrices, etc., and can select the appropriate option if it exists.", col.6, lines 38-52)
As per claim 16, Resnick in view of Bai fully discloses the limitations of claim 1, but does not explicitly disclose the feature including an event from the application
However, Rainbolt discloses:
the feature includes an event from the application. ("In particular, various approaches provide for the management of interactions between objects, and other such events, in a virtual environment, such as a gaming or artificial reality environment. Each object, or type of object, added to a gaming application can have one or more tags applied. The tags can be used to determine an interaction rule to be enforced when that object interacts with another object in the environment, or an event rule to be enforced for the occurrence of a type of event in the environment, etc. This can include, for example, determining an animation, sound, or action to trigger for the event or interaction.", col.2, lines 1-11)
As per claim 18, Resnick in view of Bai fully discloses the limitations of claim 1, but does not explicitly disclose detecting a proximity of the feature to a detection point.
However, Rainbolt discloses:
detecting the feature associated with the tag includes detecting a proximity of the feature to a detection point ("The interaction of the gameplay element, such as through proximity in space or other types of interaction, can be determined 806. In order to determine the type of actions to take for the interaction, the tags for the gameplay element and the other object of the interaction can be determined 808", col.15, lines 50-55)
As per claim 22, Resnick in view of Bai fully discloses the limitations of claim 21, but does not explicitly disclose types of tags.
However, Rainbolt discloses:
associating the feature with the tag comprises associating the tag with a type, wherein associating the tag with the action comprises associating the type with the action, and wherein associating the tag with the action comprises associating the type with the action, and wherein further triggering the device to implement the instructions comprises triggering the device to implement the action according to the type. ("The tag may be one of a set of tags defined or provided for a game, in order to provide for simplified management of the game, wherein similar objects or elements can have the same tags, or types of tags, applied, which can help to also provide for more consistent gameplay if the same types of objects or elements have the same types of interactions associated with them.", col.3, lines 24-30; " The tags can be used to determine an interaction rule to be enforced when that object interacts with another object in the environment, or an event rule to be enforced for the occurrence of a type of event in the environment, etc. This can include, for example, determining an animation, sound, or action to trigger for the event or interaction.", col.2, lines 5-11)
As per claim 24, it is a computer readable medium claim (see Resnick: [0090]) comprising substantially the same limitations as claim 22, and as such, it is rejected for substantially the same reasons.
Claims 3-5 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Summa (US 20210236944 A1)
As per claim 3, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose sending a signal configured to change a color of a light source.
However, Summa discloses:
triggering the device to implement the instructions includes sending a signal configured to cause a change in a color of a light source. ("Alternatively or in addition, the directional lighting feedback event may include modifying a color of light generated by one or more of a plurality of light sources of the lighting device. For example, an LED color may be selected based on a type of the event, such as by selecting a red or orange color for an explosion event.", 0111)
The system of Resnick in view of Bai in further view of Summa would be capable of modifying a color of a light based on a detected tag. It would have been obvious to one of ordinary skill in the art to combine the teachings of Resnick in view of Bai with those of Summa to provide the system with the ability to enhance the accessibility and immersion of the media experience by adjusting ambient light (Summa, [0111]).
As per claim 4, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose sending a signal configured to change a brightness of a light source.
However, Summa discloses:
triggering the device to implement the instructions includes sending a signal configured to cause a change in a brightness of a light source. ("Alternatively or in addition, the directional lighting feedback event may be based on the virtual distance 214. For example, an intensity of light generated by the light sources 242, 244, and 246 may be increased for a smaller virtual distance 214 (e.g., to indicate that the sound 210 appears to be close to the reference object 206)", 0110)
As per claim 5, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose sending a signal to vibrate a game controller.
However, Summa discloses:
triggering the device to implement the instructions includes sending a signal configured to vibrate a game controller. ("Alternatively or in addition, the sound metadata 208 may enable the haptic feedback device 250 to perform a directional haptic feedback event during an in-game event of one or more in-game events 204. In some examples, the haptic feedback device 250 may be configured to selectively activate one or more of the actuators 252, 254, and 256 based on an identification of the one or more in-game events 204 indicated by the sound metadata 208.", 0112)
As per claim 10, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose changing control input from a first to a second game controller.
However, Summa discloses:
the instructions include instructions for changing application control input from a first game controller to a second game controller. ("In additional examples, one or more other gameplay features can be changed, such as the positions of the AV streams, the assignment of players to the AV streams (e.g., by changing which player controls a certain character)", 0008)
It would have been obvious to one of ordinary skill in the art to combine the teachings of Resnick in view of Bai with those of Summa to provide the system with increased versatility and adaptability (Summa, [0111]).
Claims 6-7 and 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Shararay (US 20210142571 A1)
As per claim 6, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose sending a signal to change a set point of a thermostat.
However, Shararay discloses:
triggering the device to implement the instructions includes sending a signal configured to cause a change in a set point of a thermostat. (" In step 214, the processing system may send one or more signals to one or more actuators (or devices including actuators) that are capable of producing the physical effects selected in step 212. The signals may control the one of more actuators to produce physical effects in the real world that match estimated physical effects of the one or more actuators in the real world. For instance, if a physical effect comprising the raising of an ambient temperature to simulate a sudden heat surge produced by an explosion, then the processing system may send a signal to a smart thermostat to raise the ambient temperature to a specified temperature for a specified amount of time.", 0041)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick in view of Bai with those of Shararay in order to provide the media augmentation system with the ability to enhance the accessibility and immersion of the experience by changing a set point of a thermostat to influence the ambient temperature (Shararay, [0038]).
As per claim 7, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose playing the text to speech audio through a speaker.
However, Shararay discloses:
triggering the device to implement the instructions includes causing the device to play text to speech audio through a speaker ("The input/output devices 306 may include, for example, a camera, a video camera, storage devices (including but not limited to, a tape drive, a floppy drive, a hard disk drive or a compact disk drive), a receiver, a transmitter, a speaker, a display, a speech synthesizer, an output port, and a user input device (such as a keyboard, a keypad, a mouse, and the like), or a sensor.", 0055 ; Examiner Note: the presence of a speech synthesizer provides for the ability to play text to speech audio)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick in view of Bai with those of Shararay in order to provide the media augmentation system with the ability to play text to speech, thereby increasing the capabilities of the system (Shararay, [0055]).
As per claim 11, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose causing a messaging device to vibrate or display a message.
However, Shararay discloses:
triggering the device to implement the instructions includes sending a signal configured to cause a messaging device to vibrate or display a message or both vibrate and display a message. ("Similarly, if a physical effect comprises a rumble to simulate the tremors that may be caused by an explosion, then the processing system may send a signal to a rumble mechanism in a gaming chair, a gaming controller, a mobile phone, or the like to generate a rumble of a specified intensity for a specified amount of time.", 0041)
As per claim 12, Resnick in view of Bai in further view of Shararay fully discloses the limitations of claim 11, but does not disclose the messaging device being a cellular phone.
However, Shararay discloses:
the messaging device is a cellular phone. ("Similarly, if a physical effect comprises a rumble to simulate the tremors that may be caused by an explosion, then the processing system may send a signal to a rumble mechanism in a gaming chair, a gaming controller, a mobile phone, or the like to generate a rumble of a specified intensity for a specified amount of time.", 0041)
As per claim 13, Resnick in view of Bai in further view of Shararay fully discloses the limitations of claim 11.
Furthermore, Rainbolt discloses:
the messaging device is a sign board ("The device typically will include some type of display screen 906, such as a touch screen, electronic ink (e-ink), organic light emitting diode (OLED) or liquid crystal display (LCD), although devices such as portable media players might convey information via other means, such as through audio speakers.", col.16, lines 18-23 ; Examiner Note: the ‘some type of display screen’ corresponds to a sign board)
Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Dorn (US 11806630 B1).
As per claim 8, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose changing from a first button mapping profile to a second button mapping profile.
However, Dorn discloses:
the instructions include instructions for changing from a first button mapping profile to a second button mapping profile for a game controller. (" When a potential performance deficiency is identified, a user may be provided with a set of selections that allow the user to be provided with training content (e.g., overlay or other informational notification), participate in a training session or that may allow the user to change functions of the same or different gaming controller (e.g., button mapping) to account for the apparent performance deficiency. This additional training or change in controller functions may allow a user to have a more enjoyable user experience or may allow the user to perform at a higher level.", col.3, lines 5-14)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick in view of Bai with those of Dorn in order to provide the media augmentation system with the ability to enhance the accessibility and enjoyability of the experience by changing a first button mapping profile to a second, easier, button mapping profile (Dorn, [col.3, lines 5-14]).
As per claim 9, Resnick in view of Bai in further view of Dorn fully discloses the limitations of claim 8, but does not disclose changing a sensitivity of a pressure sensitive button.
However, Dorn discloses:
changing from the first button mapping profile to the second button mapping profile for the game controller includes changing a sensitivity of one or more of a mouse, a joystick, a thumbstick and a pressure sensitive button. ("Such changes may include changing a sensitivity of a steering function. A controller button may be made less sensitive when making an over steering of a vehicle less likely. As such, user selections may allow a user to change the tempo of a game or sensitivity of a gaming button to allow the user to have more fun playing the game", col.7, lines 33-49)
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Long (US 20170157512 A1).
As per claim 15, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose the feature being a map element.
However, Long discloses
the feature includes a map element from the application. ("Map data and logic analyzer 354 may analyze different areas of a game map or minimap, to determine optimal virtual camera locations that are more likely to capture exciting or critical gaming moments during a game play.", 0110)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick in view of Bai with those of Long in order to provide the media augmentation system with the ability to detect and highlight critical gaming moments (Long, [0080]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Mahlmeister (US 20230063610 A1).
As per claim 17, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose detecting when the feature is displayed on a screen.
However, Mahlmeister discloses:
detecting the feature associated with the tag includes detecting when the feature is displayed on a screen ("In one or more embodiments, the video game server 1402 can detect a current trigger during a video game and record a trigger clip. A trigger is an event in the video game that a player may find significant and/or if reviewed, improve player performance in the future. Examples of triggers can include when a player makes a kill, makes a hit, takes a hit, loss of life, fired upon, implements weapon fire, loss of life of another player, change in ammunition, etc. A player may want to review triggers so as to avoid such actions/events (e.g. loss of life, takes a hit, fired upon, etc.) or to continue the actions/events (e.g. makes a kill, makes a hit, etc.) to improve video game performance in the future. A trigger clip is a portion of the video game that is recorded that contains the trigger. As described herein, a video game may have a sliding window of game content that it continuously records portions of the video game. Once the video game server 1402 identifies a trigger, the video game server 1402 can process the recorded sliding window/recorded portion of video game content that contains the trigger as the trigger clip. Further, the trigger clip can be provided and presented to a player for the player's review.", 0275 ; Examiner Note: detecting a trigger clip equates to detecting when the feature is displayed on screen)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Resnick in view of Bai with those of Mahlmeister in order to enhance the accessibility and improve video game performance through review of trigger events (Mahlmeister, [0275]).
Claims 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Resnick (US 20230044961 A1) in view of Bai (US 20240135973 A1) in further view of Todd (US 20210344991 A1).
As per claim 19, Resnick in view of Bai fully discloses the limitations of claim 1, but does not disclose a bounding box around the perimeter of the screen.
However, Todd discloses:
the instructions include instructions for displaying a bounding box at the perimeter of a screen (“FIG. 3 shows an embodiment of a logical representation of a video container 44 being resized and positioned on a television or display device 38. The network computing device 39 displays the video container bounding box rectangle or video container representation 45. The network computing device 39 such as a mobile or PC device may act as a user interface and allow the viewer or user to resize the video container representation 45 shown on the network computing device 39 and then transmits a command over a communications network 22 to the media processing system 24.”, 0334 ; Examiner Note: the bounding box may be resized to the perimeter of the screen)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Rainbolt with those of Todd in order to provide the media augmentation system with the ability to indicate the location of a video container (Todd, [0334]).
As per claim 20, Rainbolt fully discloses the limitations of claim 1, but does not disclose a bounding box around a feature.
However, Todd discloses:
the instructions include instructions for displaying a bounding box around the feature (“The layered content stream may include outlines of recognized objects or persons of interest, labelling of recognized objects or persons of interest, highlighting of recognized objects or persons of interest, and the like.”, 0557)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Lutz (US 20210205706 A1) – discloses a method for maintaining a virtual space for a gaming environment, including repeatedly activating an event zone in the virtual space.
Dontcheva (US 20240134597 A1) – discloses a system wherein an audio track from a video is transcribed, and the transcript is parsed to identify sentences that end with a question mark. Depending on the embodiment, one or more types of questions are filtered out, such as short questions less than a designated length or duration, logistical questions, and/or rhetorical questions.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROSS MICHAEL VINCENT whose telephone number is (703)756-1408. The examiner can normally be reached Mon-Fri 8:30AM-5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Blair can be reached at (571) 270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.M.V./
Examiner, Art Unit 2196
/APRIL Y BLAIR/Supervisory Patent Examiner, Art Unit 2196
Prosecution Insights