Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 6, 8, 10-11, 13, 15-17, and 19 are under 35 U.S.C. 103 as being unpatentable over by Kulkarni (Pub No. US 20220107852 A1) in view of Zhang (Pub No. US 20230334320 A1) and further in view of Min (Pub No. US 20190050395 A1).
As per claim 1, Kulkarni teaches the claimed:
1. A computer-implemented method comprising: receiving a data file that describes a three-dimensional (3D) virtual environment, wherein the data file includes tags for attributes in the 3D virtual environment; (Kulkarni [0003]: “Content management systems, however, include a vast number of potential user actions and a vast number of potential recommendations. For example, with respect to a digital file stored on the content management system alone, a single user can create a file, view a file, share a file, edit a file, move the storage location of the file, not view the file, comment on the file, tag the file, delete the file, sync the file with a specific client device, as well as other user actions. Similarly, there are a vast number of potential recommendations to make with respect to a digital file and a particular user. Moreover, a single file may be shared with a team of users all capable of performing the large number of user actions and executing the large number of recommendations. Furthermore, the above example only relates to a single digital file. Modern content management systems provide cloud-based smart work spaces that manage not only digital content, but manage all types of services including calendar items, task lists, project spaces, different types of digital communication (e.g., IM or Video Conference), as well as other services. These services incorporate additional user actions and additional potential recommendations.” Kulkarni [0056]: “In some embodiments, to improve training and/or evaluation processes, the user activity sequence system 104 may apply a label to the raw event data and/or the series of sequential tokens 204 to indicate whether the automated workflow was actually implemented by the user account. By observing and tagging the raw event data and/or the series of sequential tokens 204, the user activity sequence system 104 can learn to better recognize cyclical/periodic patterns in user activity events for workflow automation. To do so, the user activity sequence system 104 can utilize the labels or tags (e.g., as part of a training process described below in relation to FIG. 5) in generating token sequences for populating the corpus of series of sequential tokens 510 to apply to the natural language model 202. Subsequently, in executing additional training iterations based on observed and tagged data as just described, the user activity sequence system 104 can further improve or fine-tune the natural language model 202.
Kulkarni teaches a virtual reality device that can produce a virtual environment. Kulkarni [0033]: “As shown in FIG. 1, the environment 100 includes the client devices 106. The client devices 106 can be one of a variety of computing devices, including a smartphone, tablet, smart television, desktop computer, laptop computer, virtual reality device, augmented reality device, or other computing device as described in relation to FIG. 9. Although FIG. 1 illustrates multiple client devices 106, in some embodiments the environment 100 can include just one of the client devices 106. The client devices 106 can further communicate with the server(s) 102 via the network 112. For example, the client devices 106 can receive user input and provide information pertaining to the user input to the server(s) 102.” Virtual reality involves 3D virtual environments.).
generating a set of one-hot token vectors from the tags in the data file; (Kulkarni claim 4: “The system of claim 1, further comprising instructions that, when executed by the at least one processor, cause the system to: generate, utilizing a hashing function, respective tokens for each activity event vector in the sequence of activity event vectors; and generate, based on the respective tokens, a series of sequential tokens for the sequence of activity event vectors.” Kulkarni [0076]: “At act 306, the user activity sequence system 104 can generate a feature vector embedding for each feature extracted at act 304. In some embodiments, generating a feature vector embedding for a given feature comprises converting a format of the features into a vector format. For instance, the user activity sequence system 104 may convert the following example features as follows: the “is_dir” feature from a Boolean value to an integer value (e.g., zero or one as a binary 1-dimensional vector or integer); the “byte_size” feature from a long value to a log.sub.10 value (e.g., as a one-hot encoded 10-dimensional integer vector or as the floor(log(size)) as an integer value); the “action_type” feature from an integer value (e.g., less than 80) to a base 3 value (e.g., as a four-dimensional integer vector, or else an integer if left as-is); the “file_name” feature from a text value to a filename character-distribution-based embedding (e.g., as a five-dimensional or ten-dimensional float or integer vector); the “file_ext” feature from a text value to an extension embedding (e.g., as a six-dimensional or ten-dimensional integer or float vector); and the “time_diff” feature from a seconds value to a number of months, weeks, days, and hours (e.g., as a four-dimensional integer vector).” These features are the tags.).
generating a set of metadata vectors from metadata in the data file, (Kulkarni [0020]: “One or more embodiments described herein include a user activity sequence system that utilized a natural language model to intelligently predict and provide user activity recommendations based on generalized representations of user activity sequences. Specifically, the user activity sequence system can create an activity event vector (e.g., a feature vector) for each user activity event that includes various attributes of a given activity event, for example, a timestamp, action type, device type, filename, and other file metadata. The user activity sequence system can hash the activity event vector to create an event token representing the user activity event. Accordingly, based on a vocabulary of learned event tokens and learned sequences of event tokens, the user activity sequence system can utilize a natural language model (NLM) to determine a predicted activity event. To do so, the user activity sequence system can represent each user activity event as an event token and leverage the NLM to determine a predicted event token (e.g., a next user activity event) in a sequence of event tokens. In other words, the user activity sequence system provides as input event tokens and sequences of event tokens to a trained NLM, where the NLM processes individual event tokens as “words” and sequences of event tokens as “sentences” to accurately predict a next user activity event (e.g., the next word in a sentence). Based on the predicted user activity event, the user activity sequence system can provide recommendations and suggestions, perform actions (e.g., sync a file with a particular device), and/or perform other digital actions.”).
Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Zhang, teaches the claimed:
wherein one or more metadata vectors in the set of metadata vectors correspond to one or more one-hot token vectors in the set of one-hot token vectors; (Kulkarni teaches each token vector is related to an event activity vector. Kulkarni [0007]: “Embodiments of the present disclosure provide benefits and/or solve one or more of the foregoing or other problems in the art with systems, non-transitory computer-readable media, and methods that utilize a natural language model to predict a next user activity event based on an ordered series of preceding user activity events. For example, the disclosed systems can analyze raw event data for historical user activity to create event tokens based on identified features from the raw event data. Each event token represents an activity event feature vector corresponding to an activity event associated with a set of features in the raw event data. Provided an ordered series of event tokens representing a sequence of user activity events, the system uses a trained natural language model to generate candidate sequences of activity events. Each candidate sequence of activity events includes an activity event representing a respective hypothetical (or next) user activity event within the sequence of activity events. In turn, the disclosed systems can select, as the predicted next user activity event, the most probable next user activity event represented in a candidate sequence of activity events. In accordance with the selected next user activity event, the disclosed systems can provide one or more suggestions for display within a graphical user interface of a client device or perform one or more recommended actions.” The raw event data has its own vectors. Kulkarni [0020]: “One or more embodiments described herein include a user activity sequence system that utilized a natural language model to intelligently predict and provide user activity recommendations based on generalized representations of user activity sequences. Specifically, the user activity sequence system can create an activity event vector (e.g., a feature vector) for each user activity event that includes various attributes of a given activity event, for example, a timestamp, action type, device type, filename, and other file metadata.” These are the metadata vectors that are converted to token vectors as described above. Additionally, Zhang teaches embedding vectors, which contain “metadata” and having them mapped to one-hot tokens. Thus, they correspond. Zhang [0087]: “Referring to the first processing path 1002, an embedding component 1010 breaks the input query into text tokens, e.g., corresponding to individual words, character n-grams, WordPiece fragments, byte pair encoding (BPE) fragments, etc. The embedding component 1010 can represent the text tokens as one-hot vectors. The embedding component 1010 can then map the one-hot vectors into embedding vectors, e.g., using a linear transformation layer. A position supplementing-component 1012 adds position information to each embedding vector, to produce position-supplemented embedding vectors. The position information added to each embedding vector describes its position in the sequence of text tokens. A transformer-based query-encoding component 1014 uses the same architecture shown in FIG. 2 to map the position-supplemented embedding vectors into encoder output information, which includes plural encoding output vectors. A pooling component 1016 maps the plural encoding output vectors into the encoding vector 1004 for the query, e.g., using weighted-average pooling, classification-based pooling, or some other type of aggregating function.” The mapping shows that they correspond
Embedding vector as metadata vector for combination.).
Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Min, teaches the claimed:
combining the set of one-hot token vectors and the set of metadata vectors;
(Min teaches an input feature that is a combination of a one-hot vector and real vector, which corresponds to the metadata vector. This vector can be used as an input feature. Min claim 12. “The method of claim 1, wherein the input feature comprises any one or any combination of one-hot vector, a real vector, or a function corresponding to an input layer in the autoencoder.”).
and providing a combined set of one-hot token vectors and metadata vectors as input to a deep-learning model. (Kulkarni teaches using the event tokens to be analyzed by a natural language model, which is a deep learning model. Kulkarni [0020]: “One or more embodiments described herein include a user activity sequence system that utilized a natural language model to intelligently predict and provide user activity recommendations based on generalized representations of user activity sequences. Specifically, the user activity sequence system can create an activity event vector (e.g., a feature vector) for each user activity event that includes various attributes of a given activity event, for example, a timestamp, action type, device type, filename, and other file metadata. The user activity sequence system can hash the activity event vector to create an event token representing the user activity event. Accordingly, based on a vocabulary of learned event tokens and learned sequences of event tokens, the user activity sequence system can utilize a natural language model (NLM) to determine a predicted activity event. To do so, the user activity sequence system can represent each user activity event as an event token and leverage the NLM to determine a predicted event token (e.g., a next user activity event) in a sequence of event tokens. In other words, the user activity sequence system provides as input event tokens and sequences of event tokens to a trained NLM, where the NLM processes individual event tokens as “words” and sequences of event tokens as “sentences” to accurately predict a next user activity event (e.g., the next word in a sentence). Based on the predicted user activity event, the user activity sequence system can provide recommendations and suggestions, perform actions (e.g., sync a file with a particular device), and/or perform other digital actions.” Natural language models use deep learning. The modified combined vector taught above by Min can be the vector that is input into the event token means that both the metadata and token representations can be input. Min claim 12. “The method of claim 1, wherein the input feature comprises any one or any combination of one-hot vector, a real vector, or a function corresponding to an input layer in the autoencoder.” Min claim 4: “The method of claim 3, wherein the determining of the domain comprises: generating a reconstructed feature from the embedded feature using the autoencoder, in response to the distance being greater than the threshold distance; generating a reconstruction error based on the input feature and the reconstructed feature; and determining the input sentence to be the out-of-domain sentence, in response to the reconstruction error being greater than a threshold error.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the encoding of vectors in a one-hot format and have them mapped to regular vectors that contain the features as taught by Zhang with the system of Kulkarni in order to process tagged data in one-hot format and match it to regularly formatted vectors.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the combination of one-hot vector and regular metadata vector format as taught by Min with the system of Kulkarni in order to use both representations of a feature to analyze them with machine learning.
As per claims 10 and 16, these claims are similar in scope to limitations recited in claim 2, and thus are rejected under the same rationale.
As per claim 2, Kulkarni teaches the claimed:
2. The method of claim 1, wherein the set of metadata vectors includes areas that are reserved for one or more floating-point vectors. (Kulkarni [0078]: “At act 308, the user activity sequence system 104 can concatenate feature vector embeddings to generate an activity event vector that corresponds to one activity event identified at act 302. For example, the user activity sequence system 104 may combine the feature embeddings from act 306 into a thirty-dimensional float vector (albeit other size dimensions and types of vectors are contemplated within the scope of the present disclosure). In another example, the activity event vector is a several-hundred-dimensional float vector (e.g., that accounts for a file-content embedding that represents actual digital content identified within one or more digital content items).”).
As per claims 11 and 17, these claims are similar in scope to limitations recited in claim 2, and thus are rejected under the same rationale.
As per claim 4, Kulkarni teaches the claimed:
4. The method of claim 1, wherein a metadata vector in the set of metadata vectors that is associated with a one-hot token vector that lacks metadata includes metadata features that are set to zero to indicate an absence of metadata. (Kulkarni [0076]: “At act 306, the user activity sequence system 104 can generate a feature vector embedding for each feature extracted at act 304. In some embodiments, generating a feature vector embedding for a given feature comprises converting a format of the features into a vector format. For instance, the user activity sequence system 104 may convert the following example features as follows: the “is_dir” feature from a Boolean value to an integer value (e.g., zero or one as a binary 1-dimensional vector or integer); the “byte_size” feature from a long value to a log.sub.10 value (e.g., as a one-hot encoded 10-dimensional integer vector or as the floor(log(size)) as an integer value); the “action_type” feature from an integer value (e.g., less than 80) to a base 3 value (e.g., as a four-dimensional integer vector, or else an integer if left as-is); the “file_name” feature from a text value to a filename character-distribution-based embedding (e.g., as a five-dimensional or ten-dimensional float or integer vector); the “file_ext” feature from a text value to an extension embedding (e.g., as a six-dimensional or ten-dimensional integer or float vector); and the “time_diff” feature from a seconds value to a number of months, weeks, days, and hours (e.g., as a four-dimensional integer vector).z” If each bit indicates a certain feature, it would be obvious to set all bits to 0 to show that none of the metadata features are present.).
As per claims 13 and 19, these claims are similar in scope to limitations recited in claim 4, and thus are rejected under the same rationale.
As per claim 6, Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Min, teaches the claimed:
6. The method of claim 1, wherein the deep-learning model is trained using training data that includes a plurality of one-hot token vectors that are fused to a plurality of metadata vectors. (Kulkarni teaches training an NLM with the feature vectors of both kinds. Kulkarni [0020]: “One or more embodiments described herein include a user activity sequence system that utilized a natural language model to intelligently predict and provide user activity recommendations based on generalized representations of user activity sequences. Specifically, the user activity sequence system can create an activity event vector (e.g., a feature vector) for each user activity event that includes various attributes of a given activity event, for example, a timestamp, action type, device type, filename, and other file metadata. The user activity sequence system can hash the activity event vector to create an event token representing the user activity event. Accordingly, based on a vocabulary of learned event tokens and learned sequences of event tokens, the user activity sequence system can utilize a natural language model (NLM) to determine a predicted activity event. To do so, the user activity sequence system can represent each user activity event as an event token and leverage the NLM to determine a predicted event token (e.g., a next user activity event) in a sequence of event tokens. In other words, the user activity sequence system provides as input event tokens and sequences of event tokens to a trained NLM, where the NLM processes individual event tokens as “words” and sequences of event tokens as “sentences” to accurately predict a next user activity event (e.g., the next word in a sentence). Based on the predicted user activity event, the user activity sequence system can provide recommendations and suggestions, perform actions (e.g., sync a file with a particular device), and/or perform other digital actions.” Natural language models use deep learning. The modified combined vector taught above by Min can be the vector that is input into the event token means that both the original and token representations can be input. Min claim 12. “The method of claim 1, wherein the input feature comprises any one or any combination of one-hot vector, a real vector, or a function corresponding to an input layer in the autoencoder.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the combination of one-hot vector and regular metadata vector format as taught by Min with the system of Kulkarni in order to use both representations of a feature to analyze with machine learning.
As per claim 15, this claim is similar in scope to limitations recited in claim 6, and thus is rejected under the same rationale.
As per claim 8, Kulkarni teaches the claimed:
8. The method of claim 1, wherein the tags in the data file include a tag for a name, and the method further comprises: before generating the set of one-hot token vectors, removing the tag for the name and the name from the data file. (The raw event data taught by Kulkarni implies this happens before the conversion to one-hot format. Kulkarni [0040]: “As mentioned above, the user activity sequence system 104 can predict activity events and, in response, perform an action. FIG. 2 illustrates the user activity sequence system 104 utilizing a natural language model 202 to generate a predicted activity event and perform a corresponding action in accordance with one or more embodiments. As shown, the client devices 106a-106n respectively generate raw event data 210a-210n in response to performance of one or more activity events. As used herein, the term “raw event data” refers to digital information associated with an activity event. Accordingly, the raw event data 210a-210n may include, for example, information regarding digital content item location, name (e.g., filename), size, extension, contents, access privileges (e.g., view/edit/comment privileges), author, group access, timestamps of activities, type of activity (i.e., which specific activity event), user IDs associated with the digital content item, user IDs associated with events corresponding to the digital content item, device IDs or device types associated with events, whether an action was taken on particular digital content item, etc. Raw event data can be in the form of metadata associated with a digital content item (e.g., a file or folder). In addition, raw event data can include information regarding communication between users, channels of communication (e.g., email or instant messaging), calendar items, task list management (e.g., creating tasks, reminding users of tasks, and completing tasks). In essence, any event initiated by a user within the content management system, for example, can result in raw event data that includes descriptive features corresponding to the event.” Kulkarni teaches removing unreliable information. This can include the name mentioned above. Kulkarni [0103]: “As another example of pre-processing at act 502, the user activity sequence system 104 may filter raw event data. In filtering raw event data, the user activity sequence system 104 can remove unreliable information included in the raw event data. In addition, the user activity sequence system 104 can filter out duplicative data. For example, the user activity sequence system 104 may filter out duplicative instances of a same type of activity event that occurs multiple times. Similarly, the user activity sequence system 104 may filter (e.g., select one of) multiple types of activity events identified for a single activity event. In these or other embodiments, the user activity sequence system 104 may group duplicates by a user account id, a timestamp, and a digital content item id. Additionally or alternatively, the user activity sequence system 104 may determine a group of duplicate types of activity events. In turn, the user activity sequence system 104 may ignore and/or remove such raw event data when pre-processing the raw event data at act 502.”).
As per claim 9, Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Min, teaches the claimed:
9. The method of claim 1, wherein combining the set of one-hot token vectors and the set of metadata vectors is performed using a fuse operation. (Min claim 12: “The method of claim 1, wherein the input feature comprises any one or any combination of one-hot vector, a real vector, or a function corresponding to an input layer in the autoencoder.” The examiner is treating the autoencoder as the fuse operation. It outputs a reconstructed feature, which is the combination. Min claim 13: “The method of claim 4, wherein the reconstructed feature comprises any one or any combination of one-hot vector, a real vector, or a function corresponding to an output layer in the autoencoder.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the combination of one-hot vector and regular metadata vector format as taught by Min with the system of Kulkarni in order to use both representations of a feature to analyze with machine learning.
Claims 3, 12, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kulkarni in view of Zhang and further in view of Min and further in view of Das (Pub No. US 20210263525 A1) and further in view of Chen (Pub No. US 20220188595 A1) and further in view of Caufield (Pub NO. US 20190180499 A1) and further in view of Urbach (Pub No. US 20160171743 A1).
As per claim 3, Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Das, Chen, Caulfield, Urbach, teaches the claimed:
3. The method of claim 2, wherein the one or more floating-point vectors are associated with predetermined sizes that fit data associated with one or more selected from a group of a coordinate frame, an initial size, a current size, a mesh identifier, light, image textures, a mesh part, audio, a networking service, and combinations thereof. (Kulkarni teaches different vector sizes for different information. Kulkarni [0078]: “At act 308, the user activity sequence system 104 can concatenate feature vector embeddings to generate an activity event vector that corresponds to one activity event identified at act 302. For example, the user activity sequence system 104 may combine the feature embeddings from act 306 into a thirty-dimensional float vector (albeit other size dimensions and types of vectors are contemplated within the scope of the present disclosure). In another example, the activity event vector is a several-hundred-dimensional float vector (e.g., that accounts for a file-content embedding that represents actual digital content identified within one or more digital content items).” Each different feature is taught being stored in float vectors.
Light:
Das [0071]: “At 302, example process 300 may comprise receiving a first object detection associated with a first sensor type, a second object detection associated with a second sensor type, and/or other track data and classification information associated with an object in an environment. For example, an example set of data that may be received may include, for example, lidar, radar, vision (e.g., thermal and/or visible light spectrum image(s)), and fusion pipeline object detections such as semantic segmentation probabilities for lidar and vision (e.g., in the form of float probability vectors); a metric indicating the existence of object detections associated with the track for each perception pipeline; object detection statistics for the track such as total number of voxels in lidar object detections, vision object detection confidence, radar object detection doppler ambiguity resolving status, and so on (which may be input to the combined model as continuous float values). The process 300 may additionally or alternatively receive as input classification data and/or top down segmentation probabilities (e.g., see U.S. patent application Ser. No. 15/963,833, which is incorporated in its entirety herein). Further, the received data may additionally or alternatively include geometry statistics, such as velocity, extent, fraction occluded, distance from the device including the sensors, and so on. In addition to the above data, any previously generated data associated with the track from prior cycles, ticks, or operations may also be utilized by the combined model.”
Size: Das [0023]: “An example set of data that may be input to the combined model may include, for example, lidar, radar, vision, and fusion pipeline object detections such as semantic segmentation probabilities for lidar and vision (e.g., in the form of float probability vectors); a metric indicating size with the track for each perception pipeline; object detection statistics for the track such as total number of voxels in lidar object detections, vision object detection confidence, radar object detection doppler ambiguity resolving status, and so on (which may be input to the combined model as continuous float values)...” Das [0054]: “In at least one example, the localization component 226 may include hardware and/or software to receive data from the sensor(s) 206 to determine a position, velocity, and/or orientation of the vehicle 202 (e.g., one or more of an x-, y-, z-position, roll, pitch, or yaw). For example, the localization component 226 may include and/or request/receive map(s) 238 of an environment and can continuously determine a location, velocity, and/or orientation of the autonomous vehicle within the map(s) 238. In some instances, the localization component 226 may utilize SLAM (simultaneous localization and mapping), CLAMS (calibration, localization and mapping, simultaneously), relative SLAM, bundle adjustment, non-linear least squares optimization, and/or the like to receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, and the like to accurately determine a location, pose, and/or velocity of the autonomous vehicle. In some instances, the localization component 226 may provide data to various components of the vehicle 202 to determine an initial position of an autonomous vehicle for generating a trajectory and/or for generating map data, as discussed herein. In some examples, localization component 226 may provide, to the tracking component 234, a location and/or orientation of the vehicle 202 relative to the environment and/or sensor data associated therewith.” The examiner is interpreting the initial and current size as size of an object on a screen. This can depend on its position on closeness on the screen overtime, which changes the perceived size. Thus, tracking the position and orientation determine the size. Additionally, Das teaches storing number of voxels an object occupies, which determine the size as it changes.
Texture:
Chen [0020]: “In some examples, input data 110 may represent an input with one or more channels, wherein the channels each indicate any numerical property of the input data, e.g., pixel channels, color channels, image feature channels, and/or object classification channels. In some examples, input data may comprise a tensor product of different channels, e.g., red, green, and blue (RGB) pixel data comprising RGB color data for each of a plurality of different pixels (e.g., 3×h×w channels for RGB color data for an h×w pixel image). Similarly, detected features 112 may have any suitable format, and detected features 112 may have any suitable number of channels. Non-limiting examples of detected features 112 include image feature channel data including f×h×w output channels indicating a numerical assessment of feature detection among each of f different features at each pixel of the h×w pixel image, or object classification channel data including f channels indicating a numerical assessment of object detection of f different features anywhere within the whole image. Multi-channel input data from which features may be detected may be any suitable data, in any suitable representation. Non-limiting examples of input data include images (e.g., channels for colors, image features such as edges, textures, etc., and/or classified objects), natural language data (e.g., real-valued embeddings of natural language data), intermediate outputs of a neural network or other machine learning program (e.g., feature detection data from an automatic feature detection system such as computing system 110 or any other automatic feature detector). More generally, each of input data 110 and detected features 112 may be any suitable representation of input and detected feature channels, e.g., a number, a vector, a matrix, and/or a tensor. The input data 110 and/or detected features 112 may have any suitable format for representing multi-channel numerical data, e.g., vector data, matrix data, tensor data, floating-point data, fixed-point data, integer, and/or unsigned data.” Chen teaches encoding the color channels as one-hot vectors. Chen [0060]: “In some examples, as shown in FIG. 6B, instead of using joint fusion of channels and fusion of kernel elements, it may be advantageous to utilize fusion of channels Φ(x)×.sub.1 Q without fusion of kernel elements Φ(x)×.sub.3 R. For example, fusion of channels may be more important for accuracy than fusion of kernel elements. Accordingly, L.sub.k may be reduced to 1, and L may be increased accordingly. In such case, R may be simplified as a one-hot vector [0, . . . , 0, 1, 0, . . . 0].sup.T, in which the 1 is located at the center (assuming k is an odd number), in other words, such that the tensor of the dynamic residual Φ(x)×.sub.1 Q×.sub.2 P×.sub.3 R has only one non-zero slice, which is equivalent to a 1×1 convolution. Therefore, DCD of a k×k convolution may be regarded as adding a 1×1 dynamic residual on top of a static k×k kernel convolution.”
Audio:
Caulfield teaches encoding in one-hot format. Caulfield [0136]: “FIG. 28 illustrates address selection logic in accordance with some embodiments to generate a 2-bit address output address 2824 for the SVT level specified by the sel[3:0] input 2801 from a 16-bit input address 2802. The decoding of the 4-bit sel[3:0] input 2801 may be performed in a binary-to-one-hot-decoder 2803 according to a truth table 2825. In a one hot decoder only one of the output lines l0-l6 (2804-2810) can be high at any one time and all other outputs are zero. If the level from the one-hot-decoder 2803 is high the input address pair to the current multiplexer level is selected, otherwise the output from the previous multiplexer level is selected. In the multiplexer tree the first level is controlled by decoder output l6 2806 which if high controls the multiplexer pair 2811 to select address bit pair a1:a0 if high and a3:a2 if low, with the selected pair of multiplexer inputs appearing on the output pair 2812 which is connected in turn to the l5 mux pair 2813. The second level of multiplexers 2813 is controlled by select line l5 (2805) which selects the output of the previous mux pair 2812 if high, or else if 2805 is low the a5:a4 address pair is selected. The decoding continues in a similar manner until the final multiplexer stage 2823 which produces the 2-bit output address pairs [1:0] 2824.”
Caulfield teaches audio stored in floating point vectors with specific dimensions.
Caulfield [0117]: “Turning to FIG. 18, a diagram is shown illustrating ray-casting using vectorized processing according to some embodiments with predicated execution where the processor has at least four functional units; a predicated execution (PEU) 1805, which can perform per-lane predicated operations on a vector arithmetic unit (VAU) 1807 at one of three possible arithmetic precisions (fp32, fp16 or uint8) and a scalar arithmetic unit (SAU) 1808, all of which are under the control of a branch and repeat unit (BRU) which steps through the number of voxels (depth) in the volume, one vector at a time. In some implementations, in terms of vector arithmetic, 16-bit floating-point operands may be sufficient for MP3 audio (with 32-bit floating point (fp32) being optional), thereby allowing the throughput to be doubled through the VAU 1807. In fact in principle a fractional representation with unsigned integer (e.g., u8) multiplication by reciprocal of dividend can also be utilized in many cases, allowing the VAU throughput to be almost quadrupled with respect to an fp32 mode, among other possible example implementations.
Coordinate frame:
Urbach [0053]: “The following code segment illustrates fragment shader pseudo code that can be used to render an arbitrary viewport position from layers of cube map samplers with depth information.
TABLE-US-00003 // in this simple example, a cube mesh, centered at the camera origin, is passed into the pixel shader for rasterization // The VSDO is represented by a cubemap array, which is tied to the SceneID // The ViewPortID is used to lookup ViewPort specific data for the Novel ViewPort being rendered using this function (i.e. position, FOV etc). // result is the output fragment data to be written to the render target function backdrop(int SceneID, int ViewPortID) { // the view vector can be calculated by getting a normal to the vertex position of a skybox passed from the vertex shader // or it can calculated completely in the pixel shader for the screen space fragment with a view transform set in a constant buffer and indexed with ViewPortID float3 ViewDir=normalize(vertexpos.xzy); // CubeMapArray is an array of Cubic Environment maps, with depth information // camerapos[ViewPortID] a float3 constant buffer entry storing the viewport's position in world space // probepos is a float3 constant buffer entry storing the world space origin of the cubic environment/depth map being tested for a ray intersection // Hit(float3 pos,float3 ray, inout int LayerID) is a function that performs raycasting on all loaded depth cube map samplers using the ray intersection system described // The Hit( ) function's 3rd parameter is an inout value that is set to the cube map ID where that the ray intersects by the Hit( ) function after testing is done on all depth layers in the cube map array // the LayerID can index a cube map array, which along the HitPoint return value, can lookup the proper fragment to render for this viewport int LayerID; float3 HitPoint = Hit(camerapos[ViewPortID]-probepos[SceneID],ViewDir,LayerID) result=texCUBE(CubeMapArray[LayerID],HitPoint);” This also includes camera information and coordinate frame and is stored in float vectors.
Mesh information:
Urbach [0006]: “3) In order for a user to view a scene on her local system, the 3D rendering engine must have access to the assets necessary to build the scene accurately. Such assets typically include geometry mesh data (vertices, index and attribute buffers), material and shader definitions (including procedural shaders), and texture maps. As scene complexity increases, the amount of data required to render a scene on a local client increases as well. Assets for such complex virtual worlds are often included in a download or DVD that the user must first install before the 3D rendering engine on the local client can begin rendering a scene. Therefore, the more complex and dense the virtual world, the more data the user must either have installed or download on their system. Instantaneous virtual world rendering systems that depend on streaming render assets for client-side rendering may introduce objectionable visual artifacts as a viewport is procedurally refreshed with new textures and meshes that replace placeholder versions.” Urbach [0053]: “TABLE-US-00003 // in this simple example, a cube mesh, centered at the camera origin, is passed into the pixel shader for rasterization // The VSDO is represented by a cubemap array, which is tied to the SceneID // The ViewPortID is used to lookup ViewPort specific data for the Novel ViewPort being rendered using this function (i.e. position, FOV etc). // result is the output fragment data to be written to the render target function backdrop(int SceneID, int ViewPortID) { // the view vector can be calculated by getting a normal to the vertex position of a skybox passed from the vertex shader // or it can calculated completely in the pixel shader for the screen space fragment with a view transform set in a constant buffer and indexed with ViewPortID float3 ViewDir=normalize(vertexpos.xzy); // CubeMapArray is an array of Cubic Environment maps, with depth information // camerapos[ViewPortID] a float3 constant buffer entry storing the viewport's position in world space // probepos is a float3 constant buffer entry storing the world space origin of the cubic environment/depth map being tested for a ray intersection // Hit(float3 pos,float3 ray, inout int LayerID) is a function that performs raycasting on all loaded depth cube map samplers using the ray intersection system described // The Hit( ) function's 3rd parameter is an inout value that is set to the cube map ID where that the ray intersects by the Hit( ) function after testing is done on all depth layers in the cube map array // the LayerID can index a cube map array, which along the HitPoint return value, can lookup the proper fragment to render for this viewport int LayerID; float3 HitPoint = Hit(camerapos[ViewPortID]-probepos[SceneID],ViewDir,LayerID) result=texCUBE(CubeMapArray[LayerID],HitPoint); }”
Networking service:
Urbach [0017]: “Virtual world system 20 is a network addressable system that hosts a virtual world or environment accessible to one or more users over a computer network. The virtual world system 20 may include web site and server functionality where users may request and receive identified web pages and other content over the computer network. The virtual world system may also allow users to configure and maintain personal avatars and interact with other avatars and objects hosted within a virtual environment.” The transmission of the virutla information is gonna involve the network.” It would be obvious to combine any of these together using floating point vectors and encoding them in a one-hot format.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the mesh information, viewpoint coordinate system, and networking service information stored in float vectors as taught by Urbach with the system of Kulkarni in order to process mesh information in one-hot vector format.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the light information stored in float vectors as taught by Das with the system of Kulkarni in order to store light information about graphic contexts in a vector format that can contain more detail and be processed in one-hot format by Kulkarni.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the object position and size information stored in float vectors as taught by Das with the system of Kulkarni in order to store light information about graphic contexts in a vector format that can contain more detail and be processed in one-hot format by Kulkarni.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the Audio information stored in floating point vectors as taught by Caulfield with the system of Kulkarni in order to store more detailed audio data for a virtual simulation in different vector formats including one-hot format.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the storing of texture information in floating point vectors as taught by Chen with the system of Kulkarni in order to use the vector formats taught by Chen for detailed texture calculations.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the Audio information stored in floating point vectors as taught by Caulfield with the system of Kulkarni in order to store more detailed audio data for a virtual simulation in different vector formats including one-hot format.
As per claims 12 and 18, these claims are similar in scope to limitations recited in claim 3, and thus are rejected under the same rationale.
Claims 5 is rejected under 35 U.S.C. 103 as being unpatentable over Kulkarni in view of Zhang and further in view of Min and further in view of Urbach and further in view of Chao (Yuan, Chao, Zhang, Jie, Violation Detection of Live Video Based on Deep Learning, Scientific Programming, 2020, 1895341, 12 pages, 2020. https://doi.org/10.1155/2020/1895341) and further in view of Jie (Jie Hu, Huifang Kong, Qian Zhang, Runwu Liu, Enhancing scene understanding based on deep learning for end-to-end autonomous driving, Engineering Applications of Artificial Intelligence, Volume 116, 2022, 105474, ISSN 0952-1976,https://doi.org/10.1016/j.engappai.2022.105474.)
As per claim 5, Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Min, Urbach, Chao, and Jie teaches the claimed:
5. The method of claim 1, wherein the deep-learning model outputs one or more selected from a group of a recommendation to generate content in the 3D virtual environment, an identification of a terms of service violation, a scene enhancement in the 3D virtual environment, an optimal performance setting, a streaming priority for a mesh in the 3D virtual environment, and combinations thereof. (Generating content:
Kulkarni teaches outputting recommendations from a deep-learning model, the NLM. Kulkarni [0007]: “Embodiments of the present disclosure provide benefits and/or solve one or more of the foregoing or other problems in the art with systems, non-transitory computer-readable media, and methods that utilize a natural language model to predict a next user activity event based on an ordered series of preceding user activity events. For example, the disclosed systems can analyze raw event data for historical user activity to create event tokens based on identified features from the raw event data. Each event token represents an activity event feature vector corresponding to an activity event associated with a set of features in the raw event data. Provided an ordered series of event tokens representing a sequence of user activity events, the system uses a trained natural language model to generate candidate sequences of activity events. Each candidate sequence of activity events includes an activity event representing a respective hypothetical (or next) user activity event within the sequence of activity events. In turn, the disclosed systems can select, as the predicted next user activity event, the most probable next user activity event represented in a candidate sequence of activity events. In accordance with the selected next user activity event, the disclosed systems can provide one or more suggestions for display within a graphical user interface of a client device or perform one or more recommended actions.” The recommendation can involve content in a virtual environment. Kulkarni [0167]: “Client device 1006 may be a desktop computer, a laptop computer, a tablet computer, a personal digital assistant (PDA), an in- or out-of-car navigation system, a smart TV, a virtual reality (VR) or augmented reality (AR) device, a handheld device, a wearable device, a smartphone or other cellular or mobile phone, or a mobile gaming device, other mobile device, or other suitable computing devices. Client device 1006 may execute one or more client applications, such as a web browser (e.g., Microsoft Windows Internet Explorer, Mozilla Firefox, Apple Safari, Google Chrome, Opera, etc.) or a native or special-purpose client application (e.g., Facebook for iPhone or iPad, Facebook for Android, etc.), to access and view content over network 1004.”
Terms of service:
Chao abstract: “With the rapid development of Internet technology, live broadcast industry has also flourished. However, in the public network live broadcast platform, live broadcast security issues have become increasingly prominent. The detection of suspected pornographic videos in live broadcast platforms is still in the manual detection stage, that is, through the supervision of administrators and user reports. At present, there are many online live broadcast platforms in China. In mainstream live streaming platforms, the number of live broadcasters at the same time can reach more than 100,000 people/times. Only through manual detection, there are a series of problems such as low efficiency, poor pertinence, and slow progress. This approach is obviously not up to the task requirements of real-time network supervision. For the identification of whether live broadcasts on the Internet contain pornographic content, a deep neural network model based on residual networks (ResNet-50) is proposed to detect pictures and videos in live broadcast platforms. The core idea of detection is to classify each image in the video into two categories: (1) pass and (2) violation. The experiments verify that the network proposed can heighten the efficiency of pornographic detection in webcasts. The detection method proposed in this article can improve the accuracy of detection on the one hand and can standardize the detection indicators in the detection process on the other. These detection indicators have a certain promotion effect on the classification of pornographic videos.”
Chao teaches encoding information in extracted feature vectors to tell if improper images that violate the terms of service are used. Chao introduction under fig. 1: “Pornographic recognition based on manually extracted features mainly uses different feature extraction algorithms to extract prominent features such as color, texture, and shape and uses different classifiers for the extracted features to realize pornographic image recognition. Karavarsamis et al. [33] obtained the skin color convex hull region of the image by ROI localization, calculated 15 features such as the expectation and variance of the convex hull region and the ratio of nonskinned pixels in the convex hull region in the RGB color space. Finally, the random tree forest was used for classification. Wang et al. [34] proposed a model for identifying nude images based on navel and torso features. Deselaers et al. [35] proposed to extract image information based on visual bag of words (BoW) model, feature vectors in image blocks were used as bag of words, and SVM was used to classify visual word histograms.
Pornographic image recognition based on deep learning mainly classifies pornographic images or videos through convolutional neural network (CNN) and LSTM network models. Moustaf [36] uses the NPDI [37] dataset for fine tuning on the AlexNet [38] and GoogleNet [39] models. The two models are mixed according to different thresholds, and finally the classification is achieved. The model is shown in Figure 2. Wehrmann et al. [40] proposed the Adult Content Recognition with deep neural network model, which is used to extract the features of key frames in the NPDI dataset. Long- and short-term memory networks are used to classify the extracted features. Perez et al. [41] divided video information into static information and dynamic information. Video frames are fed into a CNN model to extract static information, and optical flow and MPEG motion vectors are used to describe video motion information. The motion information is sent to a CNN to extract video motion features. Finally, support vector machines [42] are used to classify the extracted features. Pornographic image detection based on deep learning has higher accuracy than skin color and feature-based detection methods.”
Scene Enhancement:
Jie concerns deep learning recommendations as taught in the title “Enhancing scene understanding based on deep learning for end-to-end autonomous driving”. Jie abstract: “Efficient understanding of the environment is a crucial prerequisite for autonomous driving, but explicitly modeling the environment is hard to come true. In contrast, imitation learning, in theory, can arrive at the direct mapping from visual input to driving command, but the inscrutability of scene representation in imitation learning is still a challenging problem. In this paper, we propose to enhance the abstract representation of visual scene from two aspects for better scene understanding, i.e. Visual Guide path and Driving Affordances path. For Visual Guide path, we leverage semantic information as visual priors to learn the intuitive state of the environment, e.g. the spatial semantic occupation of the visual scene. For Driving Affordances path, several driving affordance indicators reflecting the relationship between environment and vehicle behavior are learned as the global guidance to guide the driving system to learn safe and efficient driving policies. With the complementarity of these two paths, a Bilateral Guide Network is designed to realize the complete mapping from visual input to driving command. Our method is evaluated on the CARLA simulator with various scenarios to demonstrate the effectiveness. Besides, comparative analyses are made with some state-of-the-art methods to justify the performance of our method in the aspect of autonomous driving.” Jie concerns storing scene information in vectors. Jie 3.1: “which is a vector representing the semantic distribution of the salient objects in the visual scene. Then, we perform a matrix multiplication between the semantic attention = map 𝐒𝐚 and the raw RGB image 𝐼 to gain abstract image 𝐼𝑎, which describes the distribution and state of salient objects in the scene. Finally, we add abstract image 𝐼𝑎 and raw RGB image 𝐼 together and feed them to the backbone network (ResNet-34 (He et al., 2016)) to obtain the visual abstract representation 𝑉 :”
Performance:
Kulkarni [0024]: “For example, the more probable a candidate sequence is, the more likely a candidate sequence is correct or at least consistent with learned patterns of user activity events within the trained NLM. In these or other embodiments, the user activity sequence system can validate a selected candidate activity event as a most probable candidate sequence (e.g., using performance metrics like click-through rate or more complex evaluations involving perplexity or cross-entropy) to determine a predicted user activity event.”Streaming Priority:
Kulkarni [0060]: “As a further example of an action at act 208, the user activity sequence system 104 can generate smart suggestions for digital content items (e.g., workspace suggestions). For example, the user activity sequence system 104 may suggest that a user account share a digital content item with another user account based on one or more of the predicted activity events 206. Additionally or alternatively, the user activity sequence system 104 may intelligently arrange (or suggest) a workspace, such as a particular location and/or ordering of digital content items (e.g., files). To illustrate, the user activity sequence system 104 may cluster and/or prioritize digital content items within a shared workspace based on a level of collaboration (e.g., according to a heuristic approach that, for instance, accounts for a number of comments, edits, etc. from different user accounts).” The user activity system would involve streaming different content. It uses the natural language model. Kulkarni [0020]: “One or more embodiments described herein include a user activity sequence system that utilized a natural language model to intelligently predict and provide user activity recommendations based on generalized representations of user activity sequences. Specifically, the user activity sequence system can create an activity event vector (e.g., a feature vector) for each user activity event that includes various attributes of a given activity event, for example, a timestamp, action type, device type, filename, and other file metadata. The user activity sequence system can hash the activity event vector to create an event token representing the user activity event. Accordingly, based on a vocabulary of learned event tokens and learned sequences of event tokens, the user activity sequence system can utilize a natural language model (NLM) to determine a predicted activity event.” Kulkarni [0061]: “For instance, the user activity sequence system 104 may prioritize (e.g., a listing order of) a more-collaborated digital content item over a less-collaborated digital content item. Similarly, the user activity sequence system 104 may prioritize a listing arrangement of a team workspace as viewable by a first user account differently than viewable by a second user account in the group. As an example, the user activity sequence system 104 may prioritize, for the first user account, one or more personal documents (e.g., notes) in the shared workspace associated with the first user account over personal documents in the shared workspace associated with the second user account.” Urbach teaches transmitting the mess geometry to client devices. This can be combined with the streaming priority of Kulkarni. Urbach [0006]: “3) In order for a user to view a scene on her local system, the 3D rendering engine must have access to the assets necessary to build the scene accurately. Such assets typically include geometry mesh data (vertices, index and attribute buffers), material and shader definitions (including procedural shaders), and texture maps. As scene complexity increases, the amount of data required to render a scene on a local client increases as well. Assets for such complex virtual worlds are often included in a download or DVD that the user must first install before the 3D rendering engine on the local client can begin rendering a scene. Therefore, the more complex and dense the virtual world, the more data the user must either have installed or download on their system. Instantaneous virtual world rendering systems that depend on streaming render assets for client-side rendering may introduce objectionable visual artifacts as a viewport is procedurally refreshed with new textures and meshes that replace placeholder versions.”
It would have been obvious to combine any group of these as outputs of the NLM trained on one-hot feature vectors of Kulkarni.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the mesh information transmitted by client devices as taught by Urbach with the system of Kulkarni in order to use machine learning to control the streaming of the Mesh information of a scene.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the deep learning output indication of a terms of service violation as taught by Chao with the system of Kulkarni in order to prevent the output and processing of images that violate some content standard including pornographic content.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the machine learning output to recommend a better scene representation as taught by Jie with the system of Kulkarni in order use machine learning to optimize the storage of an image scene for a particular aesthetic or clarity goal.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Kulkarni in view of Zhang and further in view of Min and further in view of Kouzelis (Kouzelis, Lazaros Rafail and Ourania Spantidi. “Synthesizing Play-Ready VR Scenes with Natural Language Prompts Through GPT API.” International Symposium on Visual Computing (2023).) and further in view of Trepte (Pub No. US 20250181909 A1).
As per claim 7, Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with Kouzelis and Trepte teaches the claimed:
7. The method of claim 1, wherein the data file is of a filetype is selected from a group of an extensible markup language (XML), JSON, YAML, and Universal Scene Description (USD). (XML:
KulKarni [0161]: “Additionally, communication interface 910 may facilitate communications various communication protocols. Examples of communication protocols that may be used include, but are not limited to, data transmission media, communications devices, Transmission Control Protocol (“TCP”), Internet Protocol (“IP”), File Transfer Protocol (“FTP”), Telnet, Hypertext Transfer Protocol (“HTTP”), Hypertext Transfer Protocol Secure (“HTTPS”), Session Initiation Protocol (“SIP”), Simple Object Access Protocol (“SOAP”), Extensible Mark-up Language (“XML”) and variations thereof, Simple Mail Transfer Protocol (“SMTP”), Real-Time Transport Protocol (“RTP”), User Datagram Protocol (“UDP”), Global System for Mobile Communications (“GSM”) technologies, Code Division Multiple Access (“CDMA”) technologies, Time Division Multiple Access (“TDMA”) technologies, Short Message Service (“SMS”), Multimedia Message Service (“MMS”), radio frequency (“RF”) signaling technologies, Long Term Evolution (“LTE”) technologies, wireless communication technologies, in-band and out-of-band signaling technologies, and other suitable communications networks and technologies.”
JSON:
Kouzelis pg. 20: “the generation of a JSON file that will include the following keys: object name, X, Y, Z,andfacing. We then establish preliminary constraints and provide further specifics regarding the Unity3D environment. We emphasize instructions concerning spa tial orientation. Next, we dynamically parse the database prefabs and the user-selected plane defining the room. As outlined in Sect.3.1, we can obtain real-time dimensions for each furniture item. For the room area, we store its center location and points for its four walls, assuming each room has only four walls. Door positions are also recorded, if available. Figure3 provides an API request example. In this API request, grey content is static and unchanging, while blue content is generated in real-time. User input, shown in red, is brief but triggers extensive operations, as seen in the example “Create a living room”.
YAML, USD:
Trepte [0035]: “As described herein, different files may be used for different aspects of the invention, such as for parameter files, asset lists, asset files, generation logs, and/or so forth. In some examples, any type of file may be used such as, but not limited to, a YAML file, a Universal Scene Description (USD) file, a Portable Network Graphic (PNG) file, a Printer Font Metrics (PMF) file, an image file, a text file, a Graphics Interchange Format (GIF) file, a Portable Networks Graphics (PNG) file, a Portable Document Format (PDF) file, and/or any other type of file format.” Trepte concerns processing vectors representing scenes. Trepte [0121]: “The vector processors may be programmable processors that may be designed to efficiently and flexibly execute programming for computer vision algorithms and provide signal processing capabilities. In some examples, the PVA may include a PVA core and two vector processing subsystem partitions. The PVA core may include a processor subsystem, DMA engine(s) (e.g., two DMA engines), and/or other peripherals. The vector processing subsystem may operate as the primary processing engine of the PVA, and may include a vector processing unit (VPU), an instruction cache, and/or vector memory (e.g., VMEM). A VPU core may include a digital signal processor such as, for example, a single instruction, multiple data (SIMD), very long instruction word (VLIW) digital signal processor. The combination of the SIMD and VLIW may enhance throughput and speed.” It would have been obvious to use each of these different file formats for representing scenes to be processed in different vectors formats.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the JSON file format for a virtual scene as taught by Kouzelis with the system of Kulkarni in order to facilitate processing of scenes contained in that type of file.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the YAML and USD file formats for a virtual scene as taught by Trepte with the system of Kulkarni in order to facilitate processing of scenes contained in that type of file.
Claims 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kulkarni in view of Zhang and further in view of Min and further in view of He (Pub No. US 20240362286 A1).
As per claim 14, Kulkarni alone does not explicitly teach the claimed limitations.
However, Kulkarni in combination with He teaches the claimed:
14. The system of claim 10, wherein the deep-learning model is selected from a group of a large language model, a natural language processing model, and combinations thereof. ( Kulkarni [0007]: “Embodiments of the present disclosure provide benefits and/or solve one or more of the foregoing or other problems in the art with systems, non-transitory computer-readable media, and methods that utilize a natural language model to predict a next user activity event based on an ordered series of preceding user activity events. For example, the disclosed systems can analyze raw event data for historical user activity to create event tokens based on identified features from the raw event data. Each event token represents an activity event feature vector corresponding to an activity event associated with a set of features in the raw event data. Provided an ordered series of event tokens representing a sequence of user activity events, the system uses a trained natural language model to generate candidate sequences of activity events. Each candidate sequence of activity events includes an activity event representing a respective hypothetical (or next) user activity event within the sequence of activity events. In turn, the disclosed systems can select, as the predicted next user activity event, the most probable next user activity event represented in a candidate sequence of activity events. In accordance with the selected next user activity event, the disclosed systems can provide one or more suggestions for display within a graphical user interface of a client device or perform one or more recommended actions.” He teaches using a large language model to process natural language. He [0073]: “In one embodiment, the search manager 124 may implement a generative AI locally on the server device 102. In one embodiment, the search manager 124 may access a generative AI remotely on another server device. In the latter scenario, the search manager 124 may send a natural language generation (NLG) request to a generative AI model implemented on another device over a network. The generative AI model may comprise a machine learning model that implements a large language model (LLM) to support natural language processing (NLP) operations, such as natural language understanding (NLU), natural language generation (NLG), and other NLP operations. The NLG request may request an Abstractive summary 148 of document content for search results 146. The search results 146 may comprise a subset of candidate document vectors from the set of candidate document vectors. The Abstractive summary 148 may comprise a summary of document contents for the subset of candidate document vectors in a natural language representation of a human language, such as English, French, Spanish, Korean, and so forth. The search manager 124 may receive a NLG response with the Abstractive summary 148 from the generative AI model implemented by the other server device. The search manager 124 may surface the Abstractive summary 148 and/or the candidate document vectors to a user via a graphical user interface (GUI) on a client device.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the large language model option for processing human language vectors as taught by He with the system of Kulkarni in order to allow the option to use a large language model for processing the vectors.
As per claim 20, this claim is similar in scope to limitations recited in claim 14, and thus is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS JOHN FOSTER whose telephone number is (571)272-5053. The examiner can normally be reached Mon, Fri 8:30-6. Tues-Thurs 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS JOHN FOSTER/Examiner, Art Unit 2616
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights